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Preface 


This is the second report from an evaluation of the Principal Pipeline Initiative. The Wallace Foundation is 
sponsor of the initiative and of the evaluation, which is conducted by Policy Studies Associates, Inc., and 
RAND. This report analyzes progress in leader preparation and induction in the six participating school 
districts from grant award in August 2011 through mid-2013. 


Abstract 



This second report of an ongoing evaluation of The Wallace Foundation’s Principal Pipeline Initiative de- 
scribes the six participating school districts’ activities in school leader preparation and support and analyzes 
their progress over two years. The evaluation, conducted by Policy Studies Associates and the RAND 
Corporation, is intended to inform policy makers and practitioners about the process of implementing poli- 
cies and practices for school leadership and about the results of investments in the Principal Pipeline 
Initiative. This report is based on collection and analysis of qualitative data, including semi-structured inter- 
views in spring 2013 with 113 administrators in districts and partner institutions, and surveys of novice 
principals and assistant principals. Districts are following grant requirements. For leader preparation, all are 
initiating or strengthening partnerships with external programs, and five of the six have also bolstered dis- 
trict-run programs. The district programs and varying numbers of external preparation programs are show- 
ing the desired features: selective admissions, standards-based content, problem-based learning, cohort 
models, and clinical experience. To support novice leaders, all districts have coaching arrangements of vary- 
ing duration, and all have brought in assistance to build the capacity of coaches, mentors, and supervisors. 
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Summary 

T he Wallace Foundation and six urban school districts are investing in a comprehensive strategy for 
developing a larger corps of effective principals, the Principal Pipeline Initiative. The initiative has 
four interlocking components: 

1. Leader standards to which sites align job descriptions, preparation, selection, evaluation, and support. 

2. Preservice preparation that includes selective admissions to high-quality programs. 

3. Selective hiring, and placement based on a match between the candidate and the school. 

4. On-the-job evaluation and support addressing the capacity to improve teaching and learning, with 
support focused on needs identified by evaluation. 

The participating districts are Charlotte-Mecklenburg, North Carolina; Denver, Colorado; Gwinnett 
County, Georgia; Hillsborough County, Florida; New York City, New York; and Prince George’s County, 
Maryland. They receive grants of $7.5 million to $12.5 million over five years, along with technical assis- 
tance supported by the Foundation, to improve and expand on their existing practices. They have agreed to 
adopt and implement approaches to standards-based preparation, selection, evaluation, and support for 


This report focuses on the second component and the support portion of the fourth component. Data col- 
lection for this report took place during the first two years after the initiative’s August 2011 launch. The 
districts were required to move quickly during this time to bring preparation into alignment with the initia- 
tive’s specifications, so that principals prepared according to these specifications would be leading schools 
by 2015. Drawing on findings of prior research, the Foundation had specified the following features for 
preparation and induction support: 

Preparation programs have selective recruitment and admissions based on the district’s leadership 
standards. They use a cohort model, and they offer problem-based pedagogy, content tailored to 
the district, and clinical experience. 

Novice principals and assistant principals receive on-going professional development and mentor- 
ing to strengthen their ability to improve instruction. These supports address needs identified in the 
individual’s evaluation 

Data for the evaluation were gathered through interviews with district and partner organization leaders, 
and a web-based survey of all first-, second-, and third-year principals and assistant principals in the dis- 
tricts. A total of 353 principals responded to the survey, for an overall response rate of 57 percent; the re- 
sponse rate was 88 percent in the five districts other than New York City, and 31 percent in New York. 
Among assistant principals, data are reported from the 348 respondents in the five districts other than New 
York City; the response rate in these five districts was 82 percent. 


school leaders consistent with the initiative’s specifications. 


' 
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Preservice Preparation 

The districts already had the benefit of many principals prepared according to the initiative’s specifications, 
according to the survey responses of new principals for whom almost all preservice preparation had taken 
place before the grant award. Comparing these survey responses with those gathered in a prior study that 
was highly influential in the initiative’s design (Darling-Hammond, LaPointe, Meyerson, Orr, & Cohen, 
2007), the reports of the six districts’ principals on program features and their own preparedness to lead 
were quite similar to those of graduates of exemplary preparation programs, and more favorable than 
those gathered from a national sample, in that prior study. 

Under the grant, districts launched new internal programs for principal preparation, enlarged existing pro- 
grams, and worked with partners to launch or modify external programs. Internal programs appealed to 
district decision makers because their content and delivery were more easily customized to match districts’ 
leadership standards and other district priorities; district expertise could readily be accessed; and district 
hiring managers and other key staff could see firsthand the capabilities and growth of likely future candi- 
dates. External programs also had important roles, however, stemming in part from their status as state- 
approved programs that qualified their graduates for leader certification. The bulk of state-approved pro- 
grams were in universities, but they also included alternate routes operated by nonprofit organizations. 



Four of the districts relied heavily on district-run programs for high-potential assistant principals as a final 
stage in preparation in 2012-13. For Gwinnett and Hillsborough Counties this was a continuation of existing 
policy, while for Denver and Prince George’s County it was a newer policy and the programs were newly in- 
troduced under the grant. Charlotte-Mecklenburg and New York City took a somewhat different approach, 
placing major emphasis on widening and differentiating the portfolio of options for principal preparation. 


By 2012-13, preparation programs generally showed the features specified in the grant requirements; in- 
deed, these features were reportedly present in a number of programs before the grant period began, ac- 
cording to survey and interview data. As of 2012-13, district leaders were engaged in selective recruitment 
and admissions for all the programs whose graduates were eligible to become principals. (We call these the 
“principal-qualifying programs,” in contrast to “early-stage” programs such as those designed to prepare 
assistant principals according to state or district requirements.) The use of cohort models and problem- 
based pedagogy was universal among the principal-qualifying programs. Tailoring of program content to 
the district’s context and priorities was complete in all district-run programs and was growing in all the ex- 
ternal programs at the early and principal-qualifying stages. Clinical experience was a feature of every pro- 
gram, although arrangements ranged from on-the-job assignments for working educators to full -year 
placement in a new position as apprentice leader. With respect to clinical experience, districts were working 
to increase the likelihood that participants would have authentic leadership responsibilities in schools; they 
were doing this through careful selection of the settings and through orientation and monitoring of the su- 
pervising principals. 


In addition to the challenge of ensuring a high-quality learning experience in clinical settings, another chal- 
lenge that the districts faced was that of meeting the grant’s ambitious deadline in the 2014-15 school year 
for hiring large numbers of principals who had been prepared in accordance with the initiative’s vision. 


This created an incentive for focusing on the preparation of high-potential assistant principals who might 
realistically attain principal positions within a year or two of graduation from a program. The districts 
were also strengthening their working relationships with programs whose graduates would be able to move 
into principalships later in the decade, but that effort had less urgency at this time. 

Support for Novices 

Changes in district support for new principals after the grant award included new coaching roles, larger 
rosters of mentors or coaches, and added capacity-building support for mentors, coaches, or supervisors in 
all districts. These changes were aimed at providing more intensive, individualized support to new princi- 
pals, primarily through frequent, year-long mentoring or coaching. In each district, at least 86 percent of 
first-year principals responding to the survey had coaches or mentors in 2012-13, according to the survey; 
in four of the districts, 100 percent of all first-year principals had coaches or mentors. Alignment of sup- 
port with evaluation was at an early stage, since all districts either had very recently revised their leader 
evaluation systems or were still doing so. 

Partnerships 

Districts and partners alike said that partnerships had been mutually beneficial, particularly when the part- 
ners had clear roles and responsibilities, communicated regularly about agreed-on work, and had key staff 
who were internal champions for the working relationship. For these sites, the Principal Pipeline Initiative 
brought several kinds of resources for partnership. These resources included not only funding but also 
greater access to data about program results, accompanied by technical assistance to help the districts and 
partners gather and use the data. Still, in some instances districts found that external partners were unable 
to tailor their programming as extensively as the districts might want, in particular because the partners 
had to meet state requirements. 

Future Evaluation Reports 

As the initiative and the evaluation continue, three additional reports on implementation and one on effects 
will assess the actions taken by districts and their partner institutions in this initiative. Reports to be re- 
leased in 2014 and 2016 will analyze overall implementation, identifying at each stage the structures and 
policies put in place, the results observed, and factors that have supported or impeded the sites’ progress in 
carrying out their plans. A special-focus implementation report in 2015 will address systems for evaluating 
principals’ performance. After the end of the sites’ grant period, in 2018, the final report of the evaluation 
will analyze the effects of “pipeline principals” — those prepared, selected, and supported in ways consistent 
with the initiative’s design — on important school outcomes, including growth in student achievement. 
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Introduction 


r he Wallace Foundation and six urban school districts are investing in a comprehensive strategy for 
developing a larger corps of effective principals, the Principal Pipeline Initiative. The initiative has 
four interlocking components: 

1. Leader standards to which sites align job descriptions, preparation, selection, evaluation, and support. 

2. Preservice preparation that includes selective admissions to high-quality programs. 

3. Selective hiring, and placement based on a match between the candidate and the school. 

4. On-the-job evaluation and support addressing the capacity to improve teaching and learning, with 
support focused on needs identified by evaluation. 

The Foundation made a multi-year commitment in 2011 to six sites, selected in part because they had al- 
ready launched some policies and practices consistent with these components. The districts receive grants of 
$7.5 million to $12.5 million over five years, along with technical assistance supported by the Foundation, 
to improve and expand on their existing practices. They have agreed to adopt and implement approaches 
to standards-based preparation, selection, evaluation, and support for school leaders consistent with the 
initiative’s specifications, as described below. The initiative’s theory of change holds that when an urban 
district and its partner organizations provide many talented aspiring principals with preparation, evalua- 
tion, and support following the specifications, the result will be a pipeline of principals able to improve 
teaching quality and student achievement. 

The districts are: 

Charlotte-Mecklenburg Schools, North Carolina 
Denver Public Schools, Colorado 
Gwinnett County Public Schools, Georgia 
Hillsborough County Public Schools, Florida 
New York City Department of Education, New York 
Prince George’s County Public Schools, Maryland 

The evaluation of the Principal Pipeline Initiative, conducted by Policy Studies Associates and the RAND 
Corporation, is intended to inform policy makers and practitioners about the results of investments in the 
initiative’s components, and about the process of carrying out new policies and practices around school 
leadership. Like the initiative itself, the evaluation is a multi-year endeavor, and it is designed to produce 
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different types of findings at different stages of the work. This report focuses on work that the districts and 
partner organizations are doing to prepare and support new principals and assistant principals. 


Preparation, Support, and Provider Partnerships in the Initiative Design 

The Wallace Foundation’s Principal Pipeline Initiative Request for Proposals (RFP) spelled out a number of 
ingredients for the preparation and support of novice school leaders. In the Foundation’s view, far too 
many school districts have taken a laissez-faire stance on these components of leader development: they 
allow candidates to self-select into preparation programs that vary widely in quality; then they allow newly 
appointed leaders to sink or swim in their first few years on the job. Instead, the RFP envisioned districts 
working actively on the early recruitment and selection of future leaders, shaping preparation programs to 
prepare leaders to meet district standards, and supporting novice leaders in developing the needed skills on 
the job. It also envisioned that much of this work would be done in concert with those partner institutions 
and organizations that saw the district as a valued consumer. 


We quote here in full the RFP language pertaining to preparation and support: 



“HIGH-QUALITY [PRESERVICE] TRAINING. This begins well before matriculation. University or other 
principal training programs recruit and select only the aspiring leaders with the desire and potential to be- 
come effective principals in local schools. The programs then provide sound training (the type based on re- 
search) to enough professionals to fill district hiring needs. 


A. RECRUITMENT AND SELECTION 

All training program participants meet selective admissions requirements that are based on the dis- 
trict’s leadership standards and what’s known through research about important traits of effective 
school leadership, such as deep commitment to closing the achievement gap, resilience, the ability 
to work collaboratively, and openness to continuous learning . 1 


B. STRONG [PRESERVICE] TRAINING 

Effective training is based on the district’s leadership standards and it offers, among other things, 
research-based content, problem-based pedagogy, a ‘cohort’ model (that is, the same group of stu- 
dents progress through the program together), and clinical practice. Good training programs are 
also frequently assessed to make sure they continue to prepare their graduates well for the jobs 
they will step into . 2 

Recent research has shown that collaboration between districts and training providers strengthens 
training programs, better prepares aspiring principals for their jobs, and enables districts and train- 
ing programs to provide clear feedback to one another . 3 Research is also clear that districts can in- 
fluence training programs to change and improve. 


1 These examples come from the exemplary practices cited in the Preparing School Leaders for a Changing World by Linda Darling-Hammond, Michelle LaPointe et al. 

2 Specifics may be found in Preparing School Leaders for a Changing World and in the Education Development Center's Quality Measures. 

3 Characteristics of effective partnerships are described in a Districts Developing Leaders: Lessons on Consumer Actions and Program Approaches from Eight Urban 
Districts by Margaret Terry Orr, Cheryl King et al., 2010 


“Because training is such an important component of developing good principals, we take as given that 
grantee districts will be working in tandem with one or more leader training programs of their choice, which 
we call ‘partners.’ Training may be run by universities, non-profits or the district itself. What matters is that 
the district and training providers are committed to offering effective (i.e. research-based) programs and to 
working together on shaping principals to improve student achievement in the schools they will lead. 

“ON-THE-JOB EVALUATION AND SUPPORT: The district conducts principal performance evaluations 
that assess the degree to which a new principal is developing the qualities most closely tied to improving 
teaching and student achievement. The evaluation’s findings then determine what sort of professional devel- 
opment novice principals get. 

A. EFFECTIVE EVALUATION 

Effective evaluation reflects the district’s leadership standards and measures the principal’s ability 
to improve teaching and learning, not merely the ability to manage a school building. Districts fol- 
low up their principal evaluations with constructive feedback that lets the principals understand 
their strengths, weaknesses and what’s needed to improve. 


B. PROFESSIONAL DEVELOPMENT, INCLUDING MENTORING, FOR NOVICES 

Novice principals and assistant principals with the potential to step into the number one slot re- 
ceive on-going professional development that centers on strengthening their ability to improve in- 
struction. Ideally, professional development includes strong mentoring by an experienced former or 
current principal, and the mentoring - like all aspects of professional development - emerges from 
the needs identified in the individual’s evaluation.” 



In addition to these programmatic specifications, another key feature of the Foundation’s expectation was 
that the initiative would bring about changes at scale — that substantial numbers of incoming principals 
would have experienced the kind of preparation desired. The Foundation set an ambitious timeline for 
scale-up, charging districts with filling a high proportion of their principal vacancies with graduates of 
high-quality preparation programs by January 2015, three years and five months after grant award. Each 
district already had at least one high-quality preparation program in operation, in the Foundation’s judg- 
ment, at the time of grant award. 

Research on Which the Initiative Design Was Based 

Researchers have lamented the state of leader preparation and support. After a review of preparation pro- 
grams, the then-President of Teachers College, Columbia University, Arthur Levine, delivered a famously 
scathing critique, writing: “The findings of this report were very disappointing. Collectively, educational ad- 
ministration programs are the weakest of all the programs at the nation’s education schools” (2005, p. 13). 
With respect to program admissions, he observed: “As a group, these students appear more interested in 
earning credits and obtaining salary increases than in pursuing rigorous academic studies” (p. 31). He fur- 
ther noted that the institutions as well as the students were content with undemanding programs: “...there 
was a lot of tuition to be garnered for relatively little effort on the part of institutions and students” (p. 31). 
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The Wallace Foundation has commissioned and used research on leader preparation and development in 
the years since Levine’s critique emerged. A major Wallace-supported study was particularly influential in 
shaping the aims of the initiative. It addressed both preparation and support, identifying a host of issues 
with the predominant offerings in the field (Darling-Hammond, LaPointe, Meyerson, Orr, & Cohen, 2007). 
At the preservice preparation level, the authors said, previous research indicated that programs were heavy 
on coursework and light on clinical experience; they focused excessively on management rather than in- 
structional leadership; and in general their curricula and instructional methods were not relevant to the 
work of practicing school leaders. In support for leaders, where less empirical research was available for 
these authors to review, they identified a growing consensus that programs should incorporate mentoring 
and peer support and that at this level as well, theory and practice should be combined. 

At the heart of the study was an empirical investigation of selected preparation and support programs, ini- 
tially identified through expert nominations, selected to represent a range of governance structures (e.g., 
run by a university, a district, or a state), and then examined through case studies and surveys. The re- 
searchers found that graduates of these programs, in comparison with a national sample of principals, were 
rated by themselves and by teachers in their buildings as better prepared to exercise a number of specific in- 
structional leadership skills. They also rated themselves as more likely to engage in leadership practices, 
such as use of data and development of a professional learning community, than did the national sample of 
principals. The researchers corroborated these reports through onsite observation and examination of 
school practices and achievement trends. 


identified by Darling-Hammond and colleagues as characteristic of the exemplary programs that team stud- 
ied. The RFP charged the participating districts with ensuring that programs would be standards-based; re- 
cruit and select strong candidates; organize participants into cohort groups that collaborate and progress 
together; link theory and practice through instructional approaches such as problem-based learning; and 
offer robust clinical internships or inservice learning, expertly supervised. 

Another Wallace-supported study suggested ways in which districts could ensure that programs would have 
these features. Through examination of the preparation programs in eight Wallace-funded sites, Orr and 
colleagues learned how districts used their power as consumers to shape these programs (Orr, King, & 
LaPointe, 2010). They found that districts used one or more of the following three approaches: 

As discerning customers, setting standards and competencies for their principal pool. 

As competitors, operating their own programs directly aligned with their standards and priorities. 
As collaborators, inducing local university programs to change selection criteria and customize 
program features. 

In the Principal Pipeline Initiative, The Wallace Foundation has encouraged participating districts to pursue 
these strategies and, in particular, to team up with universities rather than expecting their own “competi- 
tor” programs to provide a long-term solution. This advice was also based on findings from Orr and col- 
leagues, who found that the district-based programs in these districts were less often sustained through 
times of superintendent transition than the university-based programs. 




The Principal Pipeline Initiative RFP specified that preparation and support programs should have features 


This Evaluation Report 


The evaluation of the Principal Pipeline Initiative has a dual purpose: to analyze the processes of imple- 
menting the required components in the participating districts from 2011 through 2015; and then to assess 
the results achieved in schools led by principals whose experiences in standards-based preparation, hiring, 
evaluation, and support have been consistent with the initiative’s requirements. This report addresses imple- 
mentation, focusing on preparation and support in the participating districts, and how and why they have 
changed as of 2013. The previous report in this series describes the districts’ plans and first-year activities 
in relation to all components of the initiative (Turnbull, Riley, Arcaira, MacFarlane, &c Anderson, 2013). 

Data collection for this report took place during the first two years after the initiative’s August 2011 launch 
(Exhibit 1). During this period, the districts were required to take rapid steps to bring principal preparation 
into alignment with the initiative’s specifications, so that substantial cohorts of principals prepared accord- 
ing to these specifications could be leading schools by 2015. 


Exhibit 1: Timeline of the grants, the evaluation, and this report 


Aug 2011 

Grants 

begin 


Dec 2016 


2012 

2013 

2014 

2015 

2016 

Grants 

end 




▼ 




Period described 
in this report 


Grant activities continue 



Nov 2011 

Evaluation 

begins 


III II III III III l 

2012 2013 2014 2015 2016 2017 


Apr 2018 

Evaluation 

ends 


Apr 2012-Aug 2013 ■ 

Data collection Evaluation of Evaluation of 

for this report implementation continues effects continues 


The report tells the story of the six sites’ intensive work to bring preparation programs into alignment with 
the practices advocated by the Foundation, and to begin to improve support for novice principals along 
similar, research-based lines. It analyzes the features the districts have incorporated into programming and 
the partnerships they have forged. 

Data Sources and Methods 

This report is based on an analysis of data collected by the evaluation team from the following sources: (1) 
documents including the districts’ proposals, work plans, and progress reports; (2) semi-structured inter- 
views withll3 administrators in districts and partners (Exhibit 2) during site visits in April and May 2013, 
asking about their plans, activities, accomplishments, and challenges in relation to all facets of the initia- 
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tive; (3) an earlier round of 91 interviews with district and partner administrators in April and May 2012; 
(4) observation of and participation in cross-site meetings from 2011 through 2013, including observation 
of presentations and panel discussions by district and partner-program leaders; and (5) surveys of novice 
principals and assistant principals, which we describe next. 


Exhibit 2: Interviewees by district, 2013 



Interviewees 

District 

Total 

District 

Partner 

Charlotte-Mecklenburg 

22 

17 

5 

Denver 

21 

18 

3 

Gwinnett County 

15 

11 

4 

Hillsborough County 

18 

14 

4 

New York City 

14 

7 

7 

Prince George's County 

23 

16 

7 

Total 

113 

83 

30 


Exhibit reads: Twenty-two interviews were conducted in Charlotte-Mecklenburg, including 1 7 with district administrators and five with partner-program administrators. 



Web-based surveys addressing perceptions and experiences related to preparation, hiring, evaluation, and sup- 
port were administered in spring 2013 to all first-, second-, and third-year principals and assistant principals 
in the six districts. It is important to note that the great majority of these leaders’ experiences with prepara- 
tion predated the grant award, and thus the survey responses on that subject provide baseline data on condi- 
tions before any program changes were introduced under the grant. Their experiences and perceptions of 
support, however, are relevant in assessing the early results of the districts’ grant-funded work on support. 




Exhibit 3: Survey respondents and response rates, by district 



Principal survey 

Assistant principal survey 

District 

Total 

Response rate, 
in percent 

Number of 
respondents 

Response rate, 
in percent 

Charlotte-Mecklenburg 

43 

96% 

62 

85% 

Denver 

56 

78 

91 

81 

Gwinnett County 

39 

93 

112 

85 

Hillsborough County 

55 

90 

47 

75 

New York City 

102 

31 

N/A 

N/A 

Prince George's County 

58 

89 

36 

78 

Total 

353 

57 

348 

82 


Exhibit reads: Forty-three principals completed the survey in Charlotte-Mecklenburg, comprising 96 percent of all principals surveyed in the district. 
Note: NYC assistant principal data were not included in analyses due to a low response rate. 


A total of 353 principals responded, for an overall response rate of 57 percent. The rate was low in New 
York City, at 31 percent; across the other five districts it was 88 percent. No data are available on the char- 
acteristics of the nonrespondents. Among assistant principals, we report results only from the 348 respon- 
dents in the five districts other than New York City, where the overall response rate was 82 percent. (The 
New York response rate was 17 percent among assistant principals, and we have excluded those respon- 
dents from our analysis.) 



Additional quantitative data are drawn from the progress reports that the districts submitted to The 
Wallace Foundation in October, 2013. These reports tallied for each of the three most recent school years 
the number of principals newly hired and the numbers of graduates coming out of each preparation pro- 
gram with which the district was working closely. 


Qualitative data were coded and analyzed iteratively by the team, with codes corresponding to key features 
and expectations of the Principal Pipeline Initiative. Each tentative statement of findings was reviewed by 
site visitors for factual accuracy and revised as necessary. Project directors at the sites conducted a further 
fact-check prior to publication. 
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Future Evaluation Reporting 


As the initiative and the evaluation continue, three additional reports on implementation and one on effects 
will chronicle and assess the actions taken by districts and their partner institutions in this initiative. 
Reports to be released in 2014 and 2016 will assess overall implementation, identifying at each stage the 
structures and policies put in place, the results observed, and factors that have supported or impeded the 
sites’ progress in carrying out their plans. A special-focus implementation report in 2015 will address sys- 
tems for evaluating principals’ performance. After the end of the sites’ grant period, in 2018, the final re- 
port from the evaluation will analyze the effects of “pipeline principals” — those prepared, selected, and 
supported in ways consistent with the initiative’s design — on important school outcomes, including growth 
in student achievement. 


8 



Preservice Preparation 


R eflecting on what it had seen in its previous work on leadership and leader preparation, The 

Wallace Foundation envisioned districts not just selecting school leaders from among the individu- 
als who had acquired their state-approved credentials and who happened to apply for the job, but 
instead selectively guiding high-potential educators into leadership preparation programs. The initiative’s 
developers also wanted districts to influence or even design preparation programs to incorporate the prac- 
tices found in the handful of exemplar programs studied in depth by Darling-Hammond and colleagues. 
Further, they wanted to see the districts hiring many graduates of selective, high-quality programs as princi- 
pals by January 2015; this meant that each district had to nurture a collection of preparation programs that 
could supply well-qualified principal candidates at scale within three years of grant award. 


Since the participating districts were chosen as leaders in the field, this set of requirements did not mean a 
total abandonment of their existing programs and practices, but it did mean that each district would have 
to make some changes. We discuss here the preparation landscape across the districts before grant award 
and in 2012-13, the choices made in implementing the Principal Pipeline Initiative requirements, accom- 
plishments to date, and challenges faced. 


Before the Initiative: Pathways and Perceptions 

Before the grant period began in August 2011, each district was filling its principal vacancies with candi- 
dates who had been through the pathways then available for aspiring school leaders. We analyze the prepa- 
ration pathways found at that time, which reflected district visions of preparation as well as state policy on 
administrator certification, and the types of variation found at the district, program, and individual levels. 
Next, because our survey respondents in 2013 had completed most if not all of their preparation before 
grant award we describe the perceptions reported on our surveys by novice principals and assistant princi- 
pals as a baseline for the initiative, and we compare the principals’ perceptions with those reported for ex- 
emplary preparation programs and nationally in prior research (Darling-Hammond et ah, 2007). 

Hiring Patterns and Typical Preparation Sequences 

The grantee districts varied both in size and in the number of new principals hired in the first year of the 
grant, 2011-12 (Exhibit 4). New York City was the outlier in size, with 195 principals hired for 1,591 
schools, while each of the other districts hired fewer than 50 new principals. The number hired was small- 
est in Gwinnett, at 19. Denver had the highest rate of turnover, with about one-fourth of its principals 
newly hired in that year (26 percent), while in the other districts the percentage of principals who were 
newly hired ranged from 12 to 16 percent. 
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Exhibit 4: Principal hiring and district size, 2011-12 


District 

Number of new 
principals hired 

Number of schools 

Enrollment 

New York City 

195 

1,591 

1,041,557 

Denver 

44 

167 

81,870 

Hillsborough County 

34 

254 

195,000 

Prince George's County 

33 

205 

123,000 

Charlotte-Mecklenburg 

23 

159 

138,012 

Gwinnett County 

19 

133 

162,500 


Exhibit reads: New York City hired 195 new principals during the 2011-12 school year. It had 1,591 schools and an enrollment of 1,041,557 students. 
Source: District reports to The Wallace Foundation, October 2013 


PREPARATION PATHWAYS. The principals newly hired in 2010-11 and 2011-12 had followed pathways 
that fell into three general patterns, reflecting state and district policies (Exhibit 5). The states’ influence 
was visible in the programs through which aspiring principals qualified for state certification. All six states 
had authority to approve preparation programs for school leaders, and the bulk of the approved programs 
were university-operated in every state. The states specified some required coursework and prescribed the 
number of credit hours for these programs, which extended beyond the master’s degree. Four states — 
Colorado, Maryland, New York, and North Carolina — had also approved a few programs offering alter- 
nate routes to principal certification. 

One pathway, “university preparation only, ” was the most common one in Charlotte- 
Mecklenburg, Denver, New York, and Prince George’s County. Aspiring leaders — often classroom 
teachers — paid university tuition (sometimes with a district subsidy) for a program in which they 
could participate while working full-time. That program qualified them for state certification as an 
administrator, and our interview and survey data suggest that in practice most graduates went on 
to work as assistant principals before being tapped as principals. 

A different pathway, also available in Charlotte-Mecklenburg, Denver, New York, and Prince 
George’s County, was the “alternate route with a residency. ” In selective nonprofit programs that 
worked in collaboration with the district, aspiring leaders received fellowship support to leave their 
jobs and take a residency of a year or more as an apprentice leader, with on-the-job mentoring and 
group activities designed to develop school-leadership skills. At the end of the residency they were 
eligible for principalships. Actual placement rates as principals or assistant principals varied by 
program, district, and year. 

A third pathway, “multi-stage preparation with a district final-stage program” was found in 
Gwinnett County and Hillsborough County. Aspiring leaders first went through the university 






preparation required for initial administrator certification from the state, then served as assistant 
principals. After two or three years in that position they could be selected for a district-run, tuition- 
free program of one or two years’ duration. While continuing to work as assistant principals, they 
participated in classes and practical activities designed to inculcate the district’s philosophy and 
practice of school leadership; after graduation from that final program they were eligible to be- 
come principals. 


Exhibit 5: Principal preparation pathways before the grant 


University 


Alternate 

preparation 


route with a 

only 


residency 


Multi-stage 
with a district 
final-stage 
program 



Nonprofit 

program 

with 

residency as 
apprentice 
leader 
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In broad strokes, these three pathways depict the organizational arrangements at the start of the grant period 
in mid-2011. Within the pathways, though, there were some district-level variations. For example, the univer- 
sity preparation pathway available in Charlotte-Mecklenburg and Denver included certification programs 
custom-designed in partnerships between the districts and nearby universities: Charlotte-Mecklenburg had 
crafted the two-year Leaders for Tomorrow preparation program in partnership with Winthrop University; 
Denver had a similar partnership arrangement with the one-year Ritchie Program of the University of Denver. 

The alternate-route pathway also varied, with differences across the nonprofit organizations in program 
purposes and emphases. New Leaders prepared principals for high-poverty schools in urban districts across 
the country and thus emphasized the issues and leadership skills specific to such schools and districts. The 
New York City Leadership Academy prepared principals for New York City, with an emphasis on princi- 
pals for struggling schools. Get Smart Schools was a two-year program that placed its residents in charter 
schools, aiming to cultivate skills in innovative leadership. 

And, while Gwinnett and Hillsborough Counties shared a belief in the value of a district-run program for 
selected assistant principals who might soon become principals, their programs differed in length (one year 
in Gwinnett County, two in Hillsborough County), and the mandate for participation in the program was 
enforced more strictly in Gwinnett than in Hillsborough at that time. 

At the individual participant level, the commonalities and variation in preparation experiences among 
newly hired principals before the grant award can be seen in our survey responses. We surveyed principals 
who started their principalships in these districts in the school years 2010-11 through 2012-13. Thus, two 
of the three surveyed cohorts had already completed their preparation and had become principals by the 
time of grant award in 2011; the final cohort, who were the first-year principals of 2012-13, could have 
had at most one year of preparation after grant award, during 2011-12. 

The preparation process had typically started several years earlier for these novice principals. Among all 
principals in these three cohorts responding to the survey, the median time elapsed from the beginning of 
their leader-preparation training until they started on the job as principals was 57 months, or almost five 
years. For 25 percent of respondents, the elapsed time was eight years. Thus, for example, an aspiring 
leader might participate in a university preparation program over the course of one or more years (often 
while employed), then work in a leadership position such as assistant principal for a few years, and then 
successfully apply for a principalship. In Gwinnett or Hillsborough County, participation in a district-run 
final-stage program would ordinarily take place around the end of the time spent as an assistant principal, 
which would be some years after the start of formal preparation. 

Placement as a principal was an option after just one year of preparation for the graduates of alternate- 
route programs — New Leaders in Charlotte-Mecklenburg, New York, or Prince George’s County, or the 
New York City Leadership Academy in New York. However, not all of these programs’ graduates became 
principals at the end of a year-long program, and those who did were a distinct minority of new principals 
in the three districts. 4 


4 Our survey data indicate that graduates of these programs made up fewer than 20 percent of any of the districts' newly hired principals in any of the three years. 
Moreover, this total included graduates from any cohort of a program, not just the most recent graduating cohort. 



Our findings about the time elapsed from the beginning of preparation to appointment as a principal bring 
into sharp focus the challenge of meeting a particular Wallace Foundation requirement, that of moving 
many candidates through high-quality preparation and into principalships by January, 2015. Counting 
back 57 months from that deadline, an aspiring principal following a preparation pathway of the median 
length would have had to start his or her preparation before grant award. We discuss below the implica- 
tions of this issue for the choices and challenges found in the districts. 

The survey findings also pointed to assistant principals as a prime source of new principals. For 86 percent 
of our survey respondents, the path to the principalship included service as an assistant principal. The me- 
dian time spent in an assistant principal position was five years across all districts; it was longer, six years, 
among principals in Flillsborough County and shorter, three years, among principals in Charlotte- 
Mecklenburg and Denver. 

Individual aspiring principals varied a great deal, too, in the specific programs they had chosen at one or 
more points in the preparation process. Gwinnett and Flillsborough were the only districts that mandated 
any particular programs — their final-stage district programs — as part of principal preparation, and as of 
2011 Hillsborough allowed some exceptions to the mandate. Aspiring principals everywhere could choose 
among different universities offering programs that would qualify them for state administrator certification, 
and the individuals who became principals in a given district had been prepared for certification in a wide 
range of universities, near and sometimes far. Thus, some new principals in Charlotte-Mecklenburg and 
Denver had gone through the university program developed in concert with the district, while others had 
not. And in four districts (Charlotte-Mecklenburg, Denver, New York City, and Prince George’s County), 
aspiring principals could also choose an alternate route instead of a university for certification. 


Principals' Perceptions of Their Preparation Experiences 

Even before the start of the grant period, many of the preparation-program features that The Wallace 
Foundation called for in the initiative were already present for large numbers of future principals, accord- 
ing to the first-, second-, and third-year principals who responded to our survey (Exhibit 6). Consistent 
with the Foundation’s intention to select grantee districts that were already carrying out components of the 
initiative at least in part, these data suggest that the districts were already hiring principals who had bene- 
fited from high-quality preparation programs as The Wallace Foundation defined such programs. 

In particular, 80 percent of principals responding to the survey said that the content of their preparation 
had emphasized instructional leadership to a considerable or great extent, and the same percentage said it 
had emphasized leadership for school improvement. Case studies and problem-based learning in course- 
work were reported by 74 and 72 percent of these respondents respectively. Fewer principals, 56 percent, 
said that their leadership preparation had been tailored to the district context to a considerable or great ex- 
tent; of the features emphasized in the Principal Pipeline Initiative, this was the one with the most room for 
improvement at the start of the initiative, according to the survey data. 
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Exhibit 6: Principal perceptions of their preparation experiences 



To what extent were the following true of your formal leadership preparation training? 

Percent of principals 
saying to a considerable 
or great extent (N=3i8) 

a. Program content emphasized instructional leadership 

80% 

b. Program content emphasized leadership for school improvement 

80 

c. My interactions with peers have had a positive influence on my personal growth 

78 

d. Program gave me a strong orientation to the principalship as a career 

75 

e. 1 had coursework that included analysis and discussion of case studies 

74 

f. 1 had coursework that included analysis and discussion of field-based problems/ 
problem-based learning approaches 

72 

g. 1 had coursework that included a portfolio demonstrating learning and accomplishments 

69 

h. 1 have developed close personal relationships with other leadership students 

67 

i. 1 had coursework that included field-based projects in which 1 applied ideas in the field 

65 

j. 1 had coursework that included completion of a capstone or culminating project 

62 

k. 1 had coursework that included action research or inquiry projects 

61 

1. Program content was tailored to district context 

56 


Exhibit reads: Eighty percent of first-, second-, and third-year principals in the six districts reported that their preparation program content emphasized educational leadership to a 
considerable or great extent. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 


Principals gave generally positive assessments of the results of their preparation, from the vantage point of 
their subsequent experience as novice principals (Exhibit 7). In particular, 87 percent said that their pro- 
grams had prepared them well or very well to engage in continuous learning; 80 percent said the programs 
had prepared them to create a collaborative learning organization. They also reported that they had been 
prepared well or very well to use data (76 percent), evaluate teachers and provide feedback (75 percent), 
engage staff in decision making (74 percent), lead school change (74 percent), and plan comprehensively 
for school improvement (74 percent). It should be noted, however, that self-reports in response to these 
questions presumably reflect some upward bias. 






Exhibit 7: Principal perceptions of their preparedness 


How effectively did your leadership preparation training prepare you 
to do each of the following? 

Percent of principals 
saying well or very well 

(N=3i8) 

a. Engage in self-improvement and continuous learning 

87% 

b. Create a collaborative learning organization 

80 

c. Use data to monitor school progress, identify problems and propose solutions 

76 

d. Evaluate teachers and provide instructional feedback to support their improvement 

75 

e. Engage staff in a decision making process about school curriculum and policies 

74 

f. Lead a well-informed, planned change process for a school 

74 

g. Engage in comprehensive planning for school improvement 

74 

h. Create a coherent educational program across the school 

70 

i. Redesign school organizations to enhance productive teaching and learning 

69 

j. Design professional development that builds teachers' knowledge and skills 

64 

k. Understand how different students learn and how to teach them successfully 

58 

1. Evaluate curriculum materials for their usefulness in supporting learning 

48 


Exhibit reads: Eighty-seven percent of first, second, and third-year principals in the six districts reported that their preparation programs prepared them well or very well to engage in 
self-improvement and continuous learning. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 


As described in the introduction to this report, research on preparation programs — especially the work of 
Linda Darling-Hammond and colleagues — helped the Foundation determine what program features would 
be required under the Principal Pipeline Initiative. Our principal survey allows an exploratory comparison of 
principals’ reports on their preparation with those of the Darling-Hammond team’s two groups of respon- 
dents: graduates of the programs highlighted as exemplars, and a national comparison group of principals. 

As evidence pointing to the strength of preparation in these districts at the start of this initiative, we found 
that the reports of novice principals in these districts were fairly similar to those of graduates of the exem- 
plary programs studied by Darling-Hammond and colleagues, and more favorable in some respects than 
the reports found among that study’s national sample. This held true for these graduates’ assessments of 
program content and instructional approaches, and their self-reported preparation for specific aspects of 
leadership. In other words, the districts already appeared to enjoy the benefit of relatively strong prepara- 
tion programs. The detailed comparative findings appear in Appendix A. 

In interpreting these findings, it is important to consider that the program features identified as unusual 
strengths in the middle of the past decade may have become more widespread everywhere, not just among 
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the programs preparing principals for these districts. Because these survey instruments have not been used 
to gather nationally representative data on leader preparation in recent years, we cannot know how the 
programs serving these districts would compare with others nationally. 


Assistant Principals, Their Aspirations, and Their Preparation Experiences 

In beginning their work under the grant, the districts could see that the assistant principalship could offer a 
window for preparing a motivated group of candidates to become principals. As reported above, 86 percent 
of new principals had been assistant principals, according to the survey data. Moreover, a high proportion of 
the recently hired assistant principals aspired to become principals. In our survey of assistant principals, the 
respondents — all of whom were in their first, second, or third year in that job — overwhelmingly reported 
that they intended to go on to other jobs, usually including the job of principal. Across the board, 84 percent 
of responding assistant principals said they planned to pursue the principalship in their district. 

These ambitions reported by individuals were generally consistent with the expectations expressed by top 
district leaders, whom we interviewed about the career paths they envisioned for current assistant princi- 
pals. For example, top leaders interviewed in Gwinnett County — while acknowledging that “Individuals 
can want to be an assistant principal for their entire career, and that’s OK” — also said that the position 
“has got to be a stepping stone for your principals.” In each district, leaders recognized that assistant prin- 
cipals were their richest potential source of principals. 

Assistant principals’ perceptions of their preparation experiences (which, like those of the principals, largely 
took place before grant award) were quite similar to the perceptions that principals reported, according to 
our surveys. We asked first-, second-, and third-year assistant principals in five of the districts 5 to describe 
their leadership preparation with respect to the same features, and found that high proportions of this 
group reported that it had emphasized leadership for school improvement and instructional leadership 
(Exhibit 8). 


5 The response rate among assistant principals in New York City was too low to support analysis. 



Exhibit 8: Assistant principal perceptions of their preparation experiences 


To what extent were the following true of your formal leadership preparation training? 

Percent of APs saying to a 
considerable or great 
extent (N= 32 i) 

a. Program content emphasized leadership for school improvement 

92% 

b. Program content emphasized instructional leadership 

88 

c. My interactions with peers have had a positive influence on my personal growth 

79 

d. 1 had coursework that included analysis and discussion of case studies 

78 

e. 1 had coursework that included analysis and discussion of field-based problems/ 
problem based learning approaches 

77 

f. 1 had coursework that included a portfolio demonstrating learning and accomplishments 

76 

g. 1 had coursework that included field-based projects in which 1 applied ideas in the field 

75 

h. 1 had coursework that included action research or inquiry projects 

73 

i. 1 had coursework that included completion of a capstone or culminating project 

70 

j. 1 have developed close personal relationships with other leadership students 

70 

k. Program gave me a strong orientation to the principalship as a career 

69 

1. Program content was tailored to district context 

60 


Exhibit reads: Ninety-two percent of assistant principals reported that their preparation program content emphasized leadership for school improvement to a considerable or great extent. 
Source: Principal Survey for the “Evaluation of the Principal Pipeline Initiative,” 2013. 


Preparation Pathways in 2012-13 

By 2012-13, the paths available to aspiring principals reflected new district policies set in motion after the 
August 2011 grant award. We identify here the district visions and program features in place in 2012-13, 
and analyze the steps taken to bring programs into alignment with the features called for in the Principal 
Pipeline Initiative: selective recruitment and admissions, a cohort model, content tailored to the district, 
problem-based learning, and clinical practice. We note changes made in program procedures or design to 
conform to these grant specifications, but also identify ways in which the existing programs already met the 
specifications. 

Changes in the Program Landscape 

By 2012-13, the pathways to the principalship had changed with the introduction of new programs and 
greater district influence over program design and the selection of participants. The types of changes made 
fell into three broad categories: enhancing selective, district-run programming; adding a new district pro- 
gram as a final step before the principalship; and widening the portfolio of options among preparation pro- 
grams. Each of these approaches was embraced by two of the districts. 
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Gwinnett and Hillsborough Counties, which already had their own final-stage programs as a gateway to 
the principalship for selected assistant principals, made changes consistent with their belief in selective, dis- 
trict-run programming. Both districts were committed to a vision in which the district itself took the lead in 
preparing its future principals. District leaders explained this vision in interviews in 2012: 


The district owns those programs.... We know what's necessary to get principals prepared for 
functioning as instructional leaders in Hillsborough County. 

The people who are best able to help future school leaders understand [the principal's job] are 
the people who have the responsibility and the accountability for education in this district. 


Each of these two districts added or modified its own early-stage program as part of a multi-step pathway, 
as well as enhancing its existing final-stage program. 
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Gwinnett County instituted a more rigorous selection process for the Aspiring Leader Program that 
it offered to would-be assistant principals. It also lengthened and redesigned the clinical component 
of its final-stage Aspiring Principals Program. 

Hillsborough County launched the selective Future Leaders Academy for individuals who had their 
initial certification and hoped to become assistant principals. The district also overhauled its final- 
stage Preparing New Principals program for experienced assistant principals, revising the curricu- 
lum and adding a mentoring component, and it strengthened the policy of selecting new principals 
from among graduates of this final-stage program. 


Denver and Prince George’s County each added a new, district-run final-stage program for sitting assistant 
principals whom the district identified as especially promising candidates for the principalship. In this way 
they strengthened their control over both selection and preparation of candidates who had already moved 
through the existing preparation pathways and might become principals soon. While the new final-stage 
programs were not necessarily mandatory prerequisites for principal positions, the districts promoted them 
as a desirable step for high-potential assistant principals. A leader in one of the districts described new pro- 
gramming at the assistant principal level as an ingredient in building “a very deliberate, conscious growth 
and career-mapping development strategy for our assistant principals”: 


Denver’s Learn to Lead program provided a one-year residency at either the current school or a 
new school in which participants took on additional principal-like roles and developed leadership 
skills and perspectives that are not typically afforded assistant principals. Participants conducted 
case studies and were expected to learn from leaders in schools and non-education organizations. 

Prince George’s County’s Aspiring Leaders Program for Student Success provided a cohort of assis- 
tant principals with a structured curriculum designed to develop their school leadership capacity. 
Developed in partnership with the National Institute for School Leadership, it used instructional 
modules and trained mentors. 



Charlotte-Mecklenburg and New York City continued to rely on university preparation or an alternate route 
for principal qualification, rather than requiring or strongly encouraging an additional final-stage program. 
These districts’ visions of improved principal preparation featured a wider portfolio of options among prin- 
cipal-qualifying programs. “Multiple programs for different needs, that’s what I’m most excited about,” said 
a district interviewee in 2013. Also part of these districts’ strategies were efforts to strengthen district influ- 
ence over recruitment and selection of candidates for preparation through all programs. 

Charlotte-Mecklenburg developed new, specialized program options with two nearby universities. 

New York City, using something of an “all-of-the-above” strategy, launched new principal-qualifying 
options in universities, expanded the number of options that the district itself offered at each stage of 
preparation, and worked with its nonprofit partners who were adding early-stage programs. 

District leaders also strengthened their programs for new assistant principals, viewing these offerings as 
part of the pathway to the principalship. As just described, Gwinnett and Hillsborough Counties both stiff- 
ened the selection processes for the internal program that they offered for teachers prior to the assistant 
principalship. They positioned these programs as stepping stones required along the way to an assistant 
principal position and thus to a principalship. Hillsborough County also redesigned its Assistant Principal 
Induction Program, seeking to tighten its focus on the subset of leadership competencies that applied to as- 
sistant principals. In Charlotte-Mecklenburg, on-the-job support for novice assistant principals was re- 
designed by the same people who had designed the district’s support for novice principals, including 
partners at Queens University. 

District and Partner Programs Operating in 2012-13 

In this overview of the programs that existed in 2012-13, we differentiate between “early-stage” and “prin- 
cipal-qualifying” programs. Early-stage programs were those whose graduates had to engage in further 
preparation before applying for principalships in their districts, due to state or district requirements. 
Principal-qualifying programs, on the other hand, produced graduates who were eligible to become princi- 
pals without further formal preparation (although in practice many were likely to take a different adminis- 
trative job, such as an assistant principalship, on the way to a principalship). We also differentiate among 
programs on the basis of sector — whether the program was managed by the district itself, a university, or a 
nonprofit organization. 
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PRINCIPAL-QUALIFYING PROGRAMS. As of 2012-13 there were 20 principal-qualifying programs that 
were in operation and that the districts described as “preferred providers” 6 (Exhibit 9). New York City had 
seven operational programs at the principal-qualifying level, 7 Denver four, Charlotte-Mecklenburg four, 
Prince George’s County three, and Gwinnett and Hillsborough Counties one each. 

Eight of the 20 programs were new starts since the grant award, and the other 12 programs had undergone 
substantive revisions under the grant. Eighteen had graduating cohorts in 2013 of 30 individuals or fewer; 
the two outliers, both district-run programs in New York City, had 71 and 38 graduates respectively. In 
each of the six districts, the single largest program was either district-run or a longstanding external part- 
nership (those partners being the Ritchie Program for Denver and Leaders for Tomorrow for Charlotte- 
Mecklenburg). 

EARLY-STAGE PROGRAMS. Five districts (all but Denver) were also either operating or collaborating 
with additional, early-stage programs as of 2012-13. Although graduates of these programs could not pro- 
ceed directly to principalships, the programs were considered part of the overall leadership pathway for the 
district and in most cases were mandatory steps on that pathway by state or district policy. 

Gwinnett County required assistant principals to participate in the semester-long Aspiring Leader Program 
of the district’s Quality-Plus Leader Academy either before or just after their appointment to that position. 
Next, under state requirements, aspiring leaders would attend a university program of 30 to 39 credit 
hours to qualify for state certification. The following five universities offered such programs and had taken 
steps to align them to Gwinnett County’s priorities and leadership standards: Clark Atlanta University, 
Georgia State University, Mercer University, University of Georgia, and University of West Georgia. All 
these programs predated the grant but were modified in some respects under the grant. 


6 According to The Wallace Foundation in its instructions for providing program-level data in an annual progress report, a preferred provider program is one that 
the district "values as a high-quality program.". 

7 Two other programs were under development in collaboration with New York City but were not yet enrolling participants: one at Fordham University and the 
other at Relay Graduate School. Both planned to be open to participants in 2013-14. They are not included in this report's analyses because they had not yet fully 
taken shape at the time of data collection. 



Exhibit 9: Principal-qualifying programs operating in 2012-13, by district and program size 
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Exhibit reads: New York City hired an average of 200 principals annually in 2011-12 and 2012-13. It had two principal-qualifying programs with graduating cohorts of more than 30 in 2013, both operated by the district. Neither program 
was newly started after the grant award. 

Note: Programs newly started after grant award shown in bold. 

Source: District reports to The Wallace Foundation, October 201 3 " >U ^ ss l B ! :>oss V ssipnis A3||0d 
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Hillsborough County aspiring leaders were required to participate in three types of early-stage programs: 

(1) a university program for Florida’s Level I certification; (2) the district’s six-month Future Leaders 
Academy before appointment as an assistant principal; and (3) the district’s two-year Assistant Principal 
Induction Program. The second and third of these stages were new starts under the grant. The district had 
also begun collaboration with university certification programs at the University of South Florida and 
Nova Southeastern University. 

New York City offered one new early-stage program of its own, the Teacher Leadership Program, and also 
collaborated with the New Leaders Emerging Leaders Program. Both were aimed at developing leadership 
skills among practicing teachers who might then go on to seek administrative certification. The Emerging 
Leaders Program was also operating in Charlotte-Mecklenburg and Prince George’s County in 2012-13. 
Launched in the previous year nationally and aimed at teacher leaders and others with an interest in school 
leadership, it had become a stepping stone to the New Leaders principal-qualifying program in these districts. 

Recruitment and Selection into Preparation Programs 

Under the initiative, the districts and partners operating preparation programs worked to bring the processes 
of recruitment and selection into alignment with district expectations for principals. This meant recruiting 
candidates who appeared likely to meet these expectations and aligning formal selection criteria with district 
hiring standards, which were being redesigned under the initiative. It was also likely to mean involving dis- 
trict and school leaders in recruitment and selection. And a common practice in selection for these programs 
(existing programs as well as new ones) mirrored a practice that the Foundation encouraged in the process 
of hiring new principals: the use of performance tasks as part of the application process. 

USE OF DISTRICT STANDARDS AND DISTRICT STAFF INPUT IN RECRUITMENT AND SELEC- 
TION. All district-operated preparation programs had recruitment and selection processes that were 
aligned with the district’s most current leadership standards — and, thus, also aligned with hiring criteria. 
District-operated programs, whether at the early or principal-qualifying stage, served dual functions of 
building leadership capacity and tightening a selection process along the way to principal hiring. 

In districts that had a sequence of internal programs at the early and principal-qualifying stages, district 
staff could iteratively winnow the pool of potential leaders. In Hillsborough, for instance, each time an in- 
dividual was selected for and progressed through one of the three internal programs (two of which were 
new), the district could collect formal and informal information relevant to an eventual hiring decision. 

Districts designed program recruitment and application procedures to improve the likelihood that the partic- 
ipants selected could grow into strong principals. They hoped to focus district resources on the most promis- 
ing candidates and develop a strong program cohort. For example, New York City modified its Leaders in 
Education Apprenticeship Program to have a separate cohort for each cluster, 8 whose leaders took an ongo- 
ing role in selecting, monitoring, and supporting candidates in the program. A district official described the 
intentional involvement of the district’s cluster and network units in preparation programs, saying: 


8 New York City has about 60 networks with which schools affiliate voluntarily for many kinds of support with instructional and operational matters. Networks in 
turn are grouped into five clusters, which oversee and support the networks. Under the system's overall leader, called the chancellor, New York City also has a legally 
mandated system of superintendents (32 community superintendents, 8 high school superintendents, and 2 superintendents for special and alternative programs) 
who have formal supervisory authority over schools and their leaders. These arrangements reflect reorganizations introduced during the past decade. 



What we've tried to do is create a pathway that's not anonymous; so that people can go into 
these programs known and ... acknowledged and supported by those people who are in the 
networks or in their clusters. Because for most of the people, we're going to place them in jobs 
once they are done, and the idea is that [cluster and network managers] need to know them 
well, and they need to believe in these people. 


At the recruitment stage, the districts invested time and effort not only for their own programs but also in 
recruiting for partner programs — that is, the programs with which they had worked and that they consid- 
ered to be of high quality. The districts became more active in recruitment for partner programs under the 
grant. Participant recruitment was conducted through several methods, including open or invitation-only 
informational sessions, multi-program recruiting fairs, personal contact through district staff, and informa- 
tional media that provided side-by-side descriptions that differentiated programs. Taking an active role in 
informing candidates about their options enabled districts to communicate where programs fit in district vi- 
sions. Invitation-only events also gave districts a chance to encourage particular individuals to embark on 
leader preparation. 

Denver created new print materials to help potential applicants understand their options. For its 
two internal programs and two external partner programs, the district provided snapshot descrip- 
tions and charts with information related to eligibility, clinical experience, cohort size, and pro- 
gram focus. Denver also hosted an open recruiting fair that included presentations and 
informational tables from the district’s two internal programs and two external programs. The di- 
rector of leadership development and other district leaders made remarks and answered questions 
about internal and external programs. 

New York City developed a new online inventory of leader and teacher development programs that 
met its threshold criteria for program quality, including both district and partner programs. This 
webpage provided a one-stop overview that would help an aspiring leader find a program, among 
those pre-selected on the basis of quality, that matched his or her desired participation arrange- 
ments or substantive focus. In chart format, the page showed descriptive information on each pro- 
gram including duration, application window, and outcome in terms of degree and certification, 
with links to more detailed program pages that described curricula and required qualifications and, 
in turn, linked to the program’s own websites. This inventory had become the second-most visited 
page on the district’s website. 

For some programs, recruitment approximated an “invitation-only” model, with candidates invited to 
apply based on their nomination by school or district leaders. Programs that required district nominations 
as part of the application process tended to be internal district programs or programs in which the district 
was a strong partner. 

Three Charlotte-Mecklenburg programs, each co-designed by the district and a university, invited appli- 
cants based on district nominations. For instance, Charlotte-Mecklenburg staff worked with Leaders 
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for Tomorrow to nominate promising candidates for the program. These nominees were invited to an 
open house to learn about the program, and typically many of them applied to the program. 

Prince George’s County’s new Aspiring Leaders Program for Student Success began heavy promo- 
tional efforts for its inaugural year by soliciting recommendations of promising assistant principals 
from district instructional directors (i.e., the principal supervisors). These assistant principals were 
then emailed an invitation to an informational meeting where they could talk personally with the 
district’s director of school leadership. 

Hillsborough’s Preparing New Principals and New York City’s Leaders in Education 
Apprenticeship Program increasingly relied on nominations from higher-level district officials to 
identify individuals to recruit. While nominations were not a formal prerequisite for applying to 
these programs, they carried substantial weight and influenced the shape of the applicant pools. 

Both internal and external programs might require that applications include a recommendation from the 
applicant’s principal or a district supervisor. One district’s project director for the Principal Pipeline 
Initiative described the rationale for encouraging principals to recruit thoughtfully in the most recent ad- 
missions season: 

I wanted principals to actually help us hand pick. When you send [an invitation] out to every- 
one, you're going to have candidates that really would have no business going back for ed 
leadership, and they're going to waste their money getting this degree and never really be the 
right fit to move into school leadership. So we gave every principal five invitations. And we 
told them to hand deliver them to five teachers in their school that they think would be good. 

... Delivering them by hand, I think we did get a good crop. 


District programs had selection criteria that were aligned to district standards. For instance, Hillsborough 
County had designed candidate assessment rubrics that were based on the district’s new standards and con- 
sistent with its administrator job descriptions and evaluation criteria. 

To varying degrees, external programs’ selection criteria also reflected alignment with district standards and 
priorities. Alignment with standards could be seen in the application form, principal recommendation form, 
interview questions, and scoring rubrics. External partner programs in which the district was a co-manager, 
notably the programs working with Charlotte-Mecklenburg and the Ritchie Program in Denver, brought 
their criteria into strong alignment with the district’s leadership standards. 

At least one external program in every district included district staff in the selection process, such as in scor- 
ing, interviewing, or providing input to the final list. The new Queens University School Executive 
Leadership Academy program specified that its selection committee be split evenly between Charlotte- 
Mecklenburg and university staff. Also in Charlotte-Mecklenburg, the Leaders for Tomorrow program con- 
tinued to have selection panels that included in-district program alumni and other district principals, as 
well as district central-office staff. 



Those external programs that also served candidates from other 
districts modified their selection processes slightly to incorporate 
input from district staff. For example, the Summer Principals 
Academy of Teachers College, Columbia University, modified some 
aspects of its program, including joint selection processes with the 
district, as part of its grant-supported partnership with New York 
City. The University of South Florida, which ordinarily drew 
roughly half of its aspiring principals from Flillsborough County, 
began to get input from district staff on its applicants. 

In every district, aspiring principals who applied unsuccessfully to 
preparation programs could and did apply again, whether to the 
same program or another, and might eventually become principals. 
District leaders did not want to give false encouragement to aspi- 
rants who were very unlikely to become leaders, but they continued 
to recognize that individuals would learn and grow through a com- 
bination of professional experience and formal preparation, and 
they were patient with multi-year pathways to the principalship. 

PERFORMANCE-BASED SELECTION CRITERIA. Both before 
and after grant award, application processes often included an assessment of applicant performance on spe- 
cific tasks that simulated real-life problems and responsibilities that principals encounter. Most of these selec- 
tion processes included multiple tasks, some performed individually and others as part of an applicant 
group. For instance, in Charlotte-Mecklenburg, Queens University’s School Executive Leadership Academy 
had a half-day application event in which applicants rotated through a group simulation and discussion, 
completed a timed essay onsite, interviewed with a panel, and developed and delivered a presentation on an 
assigned topic. In all of these cases, performance tasks were rated by a panel, often composed of both district 
and program staff, using a rubric. These efforts could require a significant investment of district staff time to 
observe and score participants. 

Examples of performance tasks assessed during the application process included: 

TEACHER OBSERVATION AND FEEDBACK. Simulated teacher feedback tasks included having 
applicants observe a video of teaching, rate teacher practice using a district instrument, and pro- 
vide feedback to the teacher in writing or through role-play. 

DATA ANALYSIS. Applicants were given a set of school-level data to analyze and present. For the 
Ritchie Program in Denver, applicants discussed a dataset as a group, with panelists rating their ca- 
pabilities in communication and collaboration along with data analysis capabilities. 

TEACHER MEETING FACILITATION. The New York City Leadership Academy continued to 
use a scenario and dialogue for a simulated dysfunctional teacher inquiry team meeting. Applicants 


Examples of Recruitment and Selection 
Processes for Principal Preparation 

Invite applicants based on district nominations 
Include district staff in designing selection 
processes and rating applicants 
Develop applicant rating criteria that express the 
district's leadership standards and hiring criteria 
Use formal rating instruments and train raters 
in their use 

Simulate scenarios in which applicants perform 
key tasks of school leaders 

Draw on district data, such as evaluation scores 
and student performance 

Have applicants self-assess their performance 
at the end of the application process 
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discussed as a group what they would do as principal and were asked to come to a consensus deci- 
sion. The panel used a rubric to assess leadership potential based on what applicants said and how 
they handled the group interaction. For Hillsborough County’s new Future Leaders Academy, ap- 
plicants were given information in advance so they could prepare for and run a role play of a 
teacher professional learning community meeting. 

COMMUNITY AND PARENT COMMUNICATION. The Future Leaders Academy had appli- 
cants do a role play with a concerned parent, and Gwinnett County’s Quality-Plus Leader 
Academy applicants continued to have a simulated “in-basket” exercise in which they addressed a 
community concern. 

SCHOOL WALKTHROUGHS. Get Smart Schools in Denver continued to conduct classroom 
walkthroughs with applicants to assess their interactions in the setting and their ability to identify 
strengths and weaknesses. 

SELL-RELLECTION. Following their interviews and performance tasks, applicants to the New 
York City Leadership Academy’s new Leadership Advancement Program were asked to assess their 
performance and what they learned that day. 


Program Design and Content 

The Wallace Foundation charged districts with ensuring that programs would use cohort models, tailor 
their content to the district context, and offer problem-based pedagogy. We investigated the ways in which 
district and partner programs were working toward these features. 

COHORT MODELS HAD BECOME ALMOST UNIVERSAL AMONG THE DISTRICT AND PART- 
NER PROGRAMS, WITH THE ONLY EXCEPTIONS FOUND AMONG THE EARLY-STAGE PRO- 
GRAMS IN UNIVERSITIES. In programs with cohort models, participants are expected to attend classes 
and group activities together and complete the program at the same time as a discrete, consistent group. 
Survey responses suggest that this was already common in leader preparation before the grant, as described 
earlier in this chapter. As of 2012-13 a cohort model was part of the design of all of the principal-qualifying 
programs operated by districts or their university or nonprofit partners, and of all the early-stage programs 
operated by districts or nonprofit partners. However, most of the early-stage programs in universities 
preparing principals for Gwinnett and Hillsborough Counties instead allowed individual participants to 
progress at their own pace, as is often the case in programs of graduate education; this was true of three of 
the five programs working with Gwinnett County and both programs working with Hillsborough County. 

District and program leaders described the cohort feature as an important aspect of the experience in pro- 
grams using the model. In addition to attending classes together, cohort members might share field-based 
experiences or work on projects together. Different programs made different arrangements for cohort expe- 
riences, depending on the program’s design and priorities: 



In New Leaders, cohort members made monthly visits to other schools, including each other’s, to 
focus on particular problems of practice such as data analysis with teacher teams or providing ef- 
fective feedback on classroom instruction. In Denver’s new Learn to Lead, which brought perspec- 
tives from outside education into the curriculum, cohort members visited local businesses as well as 
schools for their leadership learning labs. 

Long-term group projects were featured in two new cohort-model programs, Queens University 
School Executive Leadership Academy and the Teachers College Summer Principals Academy. In 
the latter program, the group assignment was to design a school and present the design to district 
staff. 

New York’s Leaders in Education Apprenticeship Program had a new feature, that of grouping its 
participants into cluster-based cohorts. Each subgroup met weekly with a facilitator to discuss the 
district’s leadership competencies with reference to substantive issues that were priorities for that 
cluster. 

INCREASINGLY, PROGRAMS HAD CONTENT TAILORED TO DISTRICT STANDARDS, PRIORI- 
TIES, AND CONTEXT. Our baseline survey responses suggested that preparation programs in these dis- 
tricts could be more closely aligned with the district context: 56 percent of novice principals responded that 
“program content was tailored to district context” to a considerable or great extent, and assistant princi- 
pals responded similarly (60 percent). Among the program features assessed through the survey, these were 
relatively low ratings. 

Districts that developed their own internal programs were able to specify program content and adjust it to 
meet emerging priorities or conditions. As of 2012-13, internal district programs were increasingly aligned 
with the leadership standards, since they were led by district staff who were also involved in the develop- 
ment or revision of these standards. 

After revising and narrowing its leadership competencies, Hillsborough County reevaluated the 
content and activities of its internal preparation programs. With the help of an external consultant, 
Cross & Joftus, the district used its competency document as the framework within which it identi- 
fied specific skills and knowledge (e.g., management of leadership teams, data synthesis and analy- 
sis for decision making, clear communication with varied stakeholders) that it expected 
participants to have when entering and completing each program. From there, it revamped pro- 
gram content for Future Leaders Academy and Preparing New Principals to focus more explicitly 
on the competencies it desired in school leaders. 

In New York City, the Leaders in Education Apprenticeship Program was increasingly tailored for 
each of the district’s five clusters. In addition to having cluster-specific selection and assessment 
processes, cluster leaders could adjust the curriculum to include a focus on cluster initiatives, such 
as “instructional rounds” groups, performance-based assessment, or special education reform. 
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Some external partners had limiting factors affecting the degree to 
which they could tailor content for individual districts, although 
some tailoring was possible. Universities had to follow statewide 
requirements that specified particular courses. Moreover, university 
programs typically prepared leaders for multiple districts, each of 
which had its own priorities. New Leaders, as a national program 
operating in urban districts around the country, had identified 
areas of focus that were part of the program everywhere, such as 
developing a first-year action plan for a school, using data to drive 
instructional improvement, and building school teams and a cul- 
ture of high expectations. Nevertheless, several external programs’ 
leaders spoke to their programs’ alignment with district leadership 
standards, having designed or revised course curricula to focus on 
district priorities: 


Examples of Curricular Content 
in Preparation Programs 

District and program leaders commonly cited 
the following content areas as important in 
aligning with district priorities: 

School improvement design 
Instructional leadership and content 
knowledge, particularly in math, reading, 
and Common Core State Standards 
Teacher observation, feedback, and coaching 
Teacher evaluation and human capital 
management 

Data-driven decision making 

School climate and culture 

School law, including special education law 

Community and family relations 

Fiscal management and budgeting 


The Winthrop University Leaders for Tomorrow program, 
co-designed with Charlotte-Mecklenburg, revised its cur- 
riculum in an ongoing way to focus on the district’s prior- 
ity standards. Each of eight sessions in an introductory 
class focused on a discrete North Carolina leadership stan- 
dard, while content in other classes was refined collabora- 
tively with the district. Leaders for Tomorrow strove to 

keep content current and practical by incorporating district-specific materials and knowledge. For 
instance, when delving into leadership of a school vision and mission, the program started with the 
Charlotte-Mecklenburg district vision and then examined specific school visions and plans. A pro- 
gram participant described how program content was modified when a new superintendent identi- 
fied his priorities. 


A New Leaders interviewee explained, “In each district, we always start [planning the program 
content] with a crosswalk between New Leaders standards and the partner district’s standards.” 


Bank Street Principals Institute was founded to prepare leaders for New York City public schools, 
exclusively. While it had an institutional commitment to meeting the Educational Leadership 
Constituent Council program standards, the program worked to incorporate the New York City 
leadership competencies and district curricula. When participants focused on instructional leader- 
ship of mathematics, for instance, the program looked at New York City’s mathematics curricula. 
Furthermore, like other preparation programs in New York City, the Principals Institute became in- 
creasingly cluster-based, and program leaders responded to feedback from the cluster. 

Some external programs, old and new, offered specialized content addressing specific areas of district inter- 
est. For instance, school improvement and innovative school design were at the core of several programs. 
New Leaders in three districts and the New York City Leadership Academy in New York had always been 
intentional in developing leaders able to lead improvement of high-poverty, low-achievement schools. Get 



Smart Schools was always designed to develop leaders who could bring innovative approaches to new and 
existing schools. New programs, like Teachers College Summer Principals Academy, also included content 
focused on school design and improvement. Two new programs responded to needs that Charlotte- 
Mecklenburg had identified: Queens University School Executive Leadership Academy had a focus on lead- 
ing change, and the University of North Carolina at Charlotte focused on high school leadership. 

Each district had programs for which practitioners from inside the district provided a portion of the in- 
struction. Whether participating as core instructors or guest lecturers, they provided local practice-oriented 
context for theoretical content, shared real-time examples of application, and pushed program content to- 
ward district policies and practice. Program leaders described a range of district staff who had been in- 
volved in their programs, including superintendents, lawyers, finance staff, principal supervisors and 
mentors or coaches, human capital personnel, current principals and assistant principals, and professional 
development directors. Not surprisingly, substantive involvement of practitioners was most extensive in in- 
ternal district programs, which relied heavily on their own paid staffs. 

In Elillsborough County, internal preparation programs were managed by staff under the Assistant 
Superintendent of Administration. Content was delivered by these central staff members, principal 
coaches, principal supervisors, and other district colleagues pulled in according to expertise. 

Prince George’s County’s new internal program was developed with the help of the National 
Institute for School Leadership. In the first year, 2012-13, partner organization staff were primarily 
responsible for delivery of the curriculum. In the second year, district staff expected to co-deliver it, 
moving toward district-led delivery in subsequent years. After the program’s inaugural year, the 
district intended to make ongoing modifications to better align content with current leadership is- 
sues and practices experienced in its school buildings. 


PROBLEM-BASED PEDAGOGY WAS COMMON. Baseline survey responses suggest that principals in 
these districts usually had problem-based pedagogy as part of their preparation before the grants were 
awarded. When asked about the instructional strategies used in their preparation 
program or programs, principals responding to the survey said that their preparation 
included “analysis and discussion of case studies” (74 percent) and “analysis of 
field-based problems/problem-based learning approach” (72 percent) to a consider- 
able or great extent. Use of these strategies might have grown by 2012-13, however. 
Several district respondents described a recent or ongoing effort to focus content on 
application and shift from theory to practice, such as a Gwinnett County district 
leader who described changes in a district program: 


Examples of Problem - 
Based Pedagogy 


Case studies of leadership 
challenges, including field visits 
Simulations and role plays 
School-based projects on 
existing problems 

Entry plans for candidates 
transitioning into leader 
positions 

School design and 
improvement plans 


We would like to move toward application from theory and give more real-life 
exposure. For example, one of our sessions was on how to understand data and 
how to lead data teams. ...That is a new avenue. We make sure that whatever 
the session or topic is, there are performance activities that we know that as 
Gwinnett County Public Schools leaders that they will be expected to do. 
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With case studies, simulations, and role plays, programs offered problem-based opportunities for practi- 
tioners to grapple with issues and practice their leadership skills. Some case studies were done in person 
outside of the classroom, such as through group field visits to schools and non-education organizations. 
These allowed participants to investigate the problems and practices of leadership in the places where they 
occur, in real time, and through the eyes of practicing leaders. 

Some curricula required a more extended project grounded in a particular school. For example, in 
Charlotte-Mecklenburg, Leaders for Tomorrow charged its participants with reviewing a packet of infor- 
mation about a particular high-need school and talking with the principal, then determining the school’s 
needs and possible solutions, all while considering their own leadership strengths and weaknesses. After 
two weeks, an assessment committee (often including the school principal) interviewed each participant 
about the hypothetical intervention. School-based projects in this program and others included data analy- 
sis, facilitation of faculty meetings, action research, curriculum audits, resource allocation planning, devel- 
opment of a school-specific theory of change, and teacher coaching. 

Several principal-qualifying programs required participants to develop an “entry plan” for their transition 
into the principal role at a school. Participants in Denver’s Learn to Lead and the New York City 
Leadership Academy Aspiring Principal Program developed entry plans either for a hypothetical school or, 
in some cases, for a school in which they had been (or hoped to be) offered a principal position. Similarly, 
Hillsborough County’s Preparing New Principals participants worked on a 90-day plan that sketched goals 
and activities to help them get started in their new position as principal. 

Clinical Experience During Preparation 

Clinical experience was already part of the preservice preparation of most new principals at the start of the 
grant period, with 83 percent of first-, second-, and third-year principals responding on our 2013 survey 
that a “supervised internship/residency” was a part of their preparation. Under the grant, each district 
worked to improve the quality of the clinical experience offered in formal preparation. The new programs 
introduced under the grant all included clinical components. In working with both new and existing pro- 
grams, district leaders tried to ensure that aspiring principals would have real opportunities to learn about 
the principal’s job, as we describe here. We begin by analyzing the different arrangements for clinical expe- 
rience available to aspiring principals in the districts, then analyze what the district leaders had learned 
about meeting the challenge of providing an authentic leadership experience. 

A word on terminology may be helpful here. We use “clinical experience” as an umbrella term for the time 
that aspiring principals spent in deliberately practicing leadership skills in a school setting as part of the re- 
quirements of a formal preparation program. Some programs called the experience an “internship,” often 
(but not always) using this term when the activities were added on to or carved out of the participant’s full- 
time job as a teacher, instructional coach, assistant principal, or district staffer, among others. “Residency” 
was the term used most often for placement in a new setting for an extended period (often a full school 
year) as an apprentice leader. 

STRUCTURAL ARRANGEMENTS FOR CLINICAL EXPERIENCE. In the principal-qualifying programs, 
there were three types of arrangements: (1) participants remained in their existing position; (2) participants 



were placed in a different school for less than a year; and (3) participants had a full-year, full-time placement 
in a new position. In some districts, aspiring principals had different options depending on which program 
they joined (Exhibit 10). Half of the programs (10 of 20, including 4 operated by districts and 6 operated by 
universities) had participants remain in their current position. Seven programs, all operated by districts or 
nonprofits, placed participants in a full-time, full-year administrative position. These were salaried positions 
that gave participants a formal leadership role for the entire school year, almost always in a school other 
than the one in which the participant already worked. Finally, three programs incorporated a placement in a 
new setting for a summer or a semester. 


Exhibit 10: Principal-qualifying programs' 
clinical-experience arrangements, by district, 2012-13 




Placement in a new school 
for less than a full school 
year (e.g., summer or 

Full-year, full-time placement 

District 

Existing position 

semester) 

in a new school 


Charlotte- 

Mecklenburg 

University (Queens University School 
Executive Leadership Academy) 

University (UNC Charlotte Aspiring 
High School Principals) 

University (Winthrop 
University Leaders for 
Tomorrow) 

> 

</> 

O 
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Nonprofit (New Leaders) 
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Denver 

University (University of Denver 
Ritchie Program) 


District (Learn to Lead) 9 ^-| 

District (REDDI) 

Nonprofit (Get Smart Schools) 

Gwinnett County 


District (Quality-Plus 
Leader Academy) 


Hillsborough 

County 

District (Preparing New Principals) 



New York City 

District (Leaders in Education 
Apprenticeship Program) 

District (Assistant Principals' Institute) 

University (Bank Street) 

University (Teachers College, 
Columbia University Summer 
Principals Academy) 

Nonprofit (NYCLA 
Leadership 

Apprenticeship Program) 

Nonprofit (NYCLA Aspiring 
Principals Program) 

Nonprofit (New Leaders) 

Prince George's 
County 

District (Aspiring Leaders Program 
for Student Success) 

University (Bowie State University) 


Nonprofit (New Leaders) 


Exhibit reads: In Charlotte-Mecklenburg Schools, participants of the School Executive Leadership Academy and UNCC Aspiring Principals programs remained in their existing positions for 
their clinical practice in 2012-13. Participants of Leaders for Tomorrow were placed in a new school for a summer, and participants of New Leaders were placed in a new school for the full 
school year in 2012-13. 

Source: Interviews conducted by the evaluation team and district reports to The Wallace Foundation. 


9 Participants may choose to stay in their current school. 
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Districts and program leaders saw advantages and disadvantages to each of the arrangements. Experience in 
the existing position or a short-term clinical placement was less costly and easier to arrange than a long-term 
position, thus allowing programs to accommodate more aspiring principals. A full year of experience in a 
new setting might offer greater opportunity for ongoing, authentic leadership responsibilities but brought the 
cost of an extra position. Full-year placements could also disrupt the staffing equilibrium in two schools, 
both the school that the participant left (where his or her responsibilities had to be reassigned) and poten- 
tially also in the school where he or she was placed (where other members of the leadership team might have 
their jobs redesigned to accommodate the newcomer). Some program leaders cited the advantages of short- 
term placements as bringing the advantage of a new context while limiting cost and disruption. 

AUTHENTIC LEADERSHIP OPPORTUNITIES DURING CLINICAL PRACTICE. Prior research points 
to qualitative differences in clinical experiences, particularly whether the participants have an apprenticeship 
in which they actively make decisions of consequence that a school leader would make (Darling-Hammond 
et al., 2007, p. 72). The districts participating in the Principal Pipeline Initiative worked with their in-house 
and partner programs to shape participant experiences to ensure authentic leadership opportunities. 

Every early-stage program operated by a district or in partnership with a district included a clinical practice 
component, always in the participant’s existing job. University programs typically required a set number of 
hours to be spent in leadership-related tasks of the type shown as examples in the text box. 

A leader in a district that was building a partnership with a local university described their discussions 
around clinical experience in an early-stage program: 


There's just been real excitement about re-visioning the internship in the [university] program, 
for example. They've just had, and they will acknowledge this, a weak internship experience. 
The students acquire something like 400 hours, but [program leaders] almost don't care how 
people get those hours. We're saying, it's actually really important how people get those 
hours.... The experience really has to be the job. What is real experience? 


Examples of Leadership Tasks 
Participants Conducted from their 
Existing Positions 

Lead inquiry team or professional learning com- 
munity meetings 

Develop programming, such as after-school , en- 
richment, and remediation programs 
Serve on the principal's cabinet, participating in 
meetings on school-wide issues 
Conduct classroom observations and feedback 
(although evaluative observations may require for- 
mal administrator status) 


Develop a plan to address a specific weakness in 
student achievement, such as for a grade-level or 
student sub-group 

Advance school-level adoption of curricula or in- 
structional strategies, such as those related to 
Common Core State Standards 
Analyze and present data at the student, sub- 
group, grade, and school levels in a way that can 
lead to specific actions 

Provide teacher professional development 
Communicate with parents and the community 
Participate in budget planning 



Ultimately, the participant’s role and responsibilities had to be approved and supported by that school’s 
principal. Respondent descriptions suggested that some supervising principals readily included participants 
in meaningful leadership activities, while others delegated tasks that might be necessary (e.g., bus and 
lunchroom duty, inventory logs, assessment logistics) but provided limited opportunities for growth. We 
heard of participants who spent significant time shadowing the principal or did not experience a change in 
responsibility from their previous position. 

Supervising principals had reasons for their hesitancy in assigning crucial leadership tasks to preparation 
program participants. They were accountable for the performance of their schools, and they might worry 
about turning over some responsibility for their students and staff. In some cases, supervising principals 
paid for a portion of intern or resident salaries from their school budgets, which led them to assign tasks 
that served their or the school’s administrative needs. One district respondent explained, 

When you are asking a school to pay, thinking as a principal, then I own the position. ...As a 
principal, you are faced with the tension of what is best for my school and what does this per- 
son need to grow as a leader, and how closely these align varies. We tried to be intentional 
about that... [but principals could think,] "here is a low-cost way to get good staff." 


A Gwinnett County leader in the Quality-Plus Leader Academy explained how the academy counseled the 
supervising principals: 
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We caution principals against considering residents as additional support. This is a person that 
you have agreed to take and you are accountable and responsible to develop them and push 
them. Do not treat them as an additional AP or we will stop [the placement]... No, there may 
be some value in shadowing, but at this point these individuals are developing on their own 
career path. 


Gwinnett County leaders also assessed and revised the clinical experience offered. In particular, they added 
structures such as plans and closer monitoring of participant progress: 


We have reinstituted some structure to our residency. Our concern is that if we are not struc- 
tured, it would be difficult to monitor and measure progress. So now we are insisting that each 
participant have an individualized residency plan — much like an IEP [individualized education 
plan] for a child. ... Who, what, when, where, and why has to be committed to paper. And a 
plan for how we are going to monitor. How often will we visit with them? What will we ask 
those individuals? When are we are talking with them to make sure that they are hitting their 
targets? Who is going to measure it, and how are we monitoring? 


MENTORING IN CLINICAL PRACTICE. All programs involved the host principal as a supervisor of the 
clinical experience, and nearly all had explicit expectations for that host principal to serve as a mentor for 
the program participant. 
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There was broad agreement among district and program leaders that the mentor principal in the host 
school was crucial for a high-quality clinical experience with authentic leadership tasks. To address the 
challenge of providing these types of experiences for aspiring leaders, several districts’ respondents were 
turning toward improving their communication with and training of mentor principals. One district leader 
prefaced a description of the plans by explaining, 


I haven't done as good of a job at getting their principals in and telling them, "[Leader devel- 
opment] is your responsibility. This is what this candidate needs to learn. You need to make 
sure they have actual experiences and actually prescribe those things to them and do a better 
job developing them." 


A New York district leader expressed similar thoughts, suggesting that districts must be persistent in help- 
ing principals embrace a role in developing leaders and providing them with real growth opportunities. 


We continue to push at our leaders to understand that a critical part of their work as a leader is 
capacity building. ... Capacity building is two things. It's building the capacity of somebody to 
do a better job in their current position and to also create opportunities for them to apply 
some of those skill sets to more challenging situations that would make them see that they 
could move to a new position and be successful. ... That's changing people's mindset, and 
we've been clear and deliberate about that in the work that we're doing in all our programs. 
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Program and district leaders also worked to identify host principals who had the needed capacities in both 
school leadership and mentoring, as well as commitment and time for serving as a mentor. Denver differen- 
tiated the roles of mentor principals in three of its principal-qualifying programs on a mentor application 
form. A program leader in Prince George’s County expressed a sentiment we heard from several other re- 
spondents, saying: 


What we are learning is that because a person is a great principal getting great results that 
does not mean they are a great mentor. You have to be willing to share power with the resi- 
dent. You have to be willing to have someone in your ear asking thousands of questions for 
the first two months. 


Another district added extra support for program participants, no longer just relying on the sitting princi- 
pal. A program leader explained: 


The leader that's in their school serves as their mentor. The difference now between last year 
and this year is I've also assigned a principal coach to them in their second year of the program. 
Because I was concerned by some of the people that were getting in, and is their principal the 
best person to be modeling instructional leadership? And the answer in some cases is no. 



Every district provided training and support for the principals who supervised clinical experience in princi- 
pal-qualifying programs. Examples included: 

Orientation and training events intended to clarify the program and the mentor’s roles and tasks. 

Professional development to build the capacity of mentor principals, such as in coaching and talent 
development. 

Group events that convened mentors and participants to advance their relationships and shared work. 
Support to mentors from a cadre of coaches or district leaders. 

Some programs used combinations of strategies for supporting mentors: 


Learn to Lead in Denver provided mentor principals with coaches who were also charged with 
working with participants in their residency. With a heavy focus on supporting leader development 
rather than managing resident tasks, the district’s program leader met as often as monthly with 
each mentor and participant pair to support the residency experience. This close monitoring of in- 
dividual residency experiences was said to have the added benefit of enabling district staff to track 
progress of participants who would be applying for principal positions in the spring of their resi- 
dency year. 

In New York City’s district-run Leaders in Education Apprenticeship Program, the supervising 
principals were brought together regularly: “Lour times a year, we invite them to engage in profes- 
sional development. One is to inform them about the expectations of mentoring, to talk together as 
a professional learning community, to talk about some of the challenges, [such as,] ’I have a small 
school. I want to expose my apprentice to opportunities in leadership. However, how do I release 
that person without sacrificing some other component of the school?’ And so mentor principals 
gather together and they solve those challenges together. We wanted to promote that and engage 
them in that conversation.” 
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Challenges and Problem Solving in Preparation Programs 

Authentic Leadership Experience in Preparation 

As just discussed, a persistent challenge for clinical experiences was that of ensuring that the participant 
would gain an authentic leadership experience, engaging in the roles and responsibilities of the position for 
which he or she would apply next. Districts and programs might struggle to find enough clinical place- 
ments that fit district, program, and participant needs. Districts and programs often had to work to recruit 
mentor principals, yet they also hoped to be selective and confident that mentors were highly capable and a 
good fit with individual participants. Some districts also intended to have placement schools mirror the 
types of schools where they anticipated forthcoming leadership vacancies, although their capacity to project 
vacancies was limited. 
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Practical challenges accompanied the practice of placing participants outside their existing schools for a full 
year: the participant was both leaving a position in one school and taking on a position in another school, 
with ripple effects on the other members of the administrative staff in both of those schools. Arranging cov- 
erage for the responsibilities of the departing participant in his or her former school sometimes required 
fast planning in district offices. And melding the new arrival into a smoothly functioning administrative 
team at the new site — sometimes including one or more assistant principals at that site who might them- 
selves be eager to advance to principalships — could pose interpersonal as well as organizational challenges. 

Program and district leaders saw advantages and disadvantages to every arrangement for clinical experi- 
ence. They were not convinced that the longest-term placements offered the best preparation. Instead, they 
were working to improve the quality of the experience, whatever the arrangements for duration and loca- 
tion might be. 

Moving to Scale Rapidly 

In its early communications with participating districts, The Wallace Foundation had emphasized the ur- 
gency of putting the desired preparation program features in place — and, in particular, of hiring new princi- 
pals whose preparation had these features. The Foundation required that programs with these desired 
features should prepare a high proportion of the new principals appointed by January 2015. District lead- 
ers knew that early-stage programs started or revamped in 2012-13 would be of no help in meeting this re- 
quirement because their 2013 graduates would not have the qualifications required for principals and thus 
would be very unlikely to become principals within about 18 months of graduation. The same problem ap- 
plied to any principal-qualifying programs whose graduates typically spent at least two years as assistant 
principals, which was a common pattern among such programs. Thus the impending deadline of the 2014- 
15 school year created a sense of urgency around principal-qualifying programs whose graduates would be 
especially well equipped to step directly into a principalship. 

An appealing solution to this problem was to focus on the later stages of the preparation sequence. In the 
context of the Wallace timeline, it is not surprising that Denver and Prince George’s County initiated new 
programs for selected assistant principals in 2012-13, or that Gwinnett County, Flillsborough County, and 
New York City placed a priority on improving the programs they were already using to prepare seasoned 
assistant principals for the transition to a principalship. Because all these programs were district-run, their 
selection procedures and content offerings could be quickly organized or enhanced to incorporate the 
needed features and to produce new principals. 

Districts also worked to improve early-stage programs, believing that such programs would have long-term 
value. However, with several years likely to elapse between an aspiring principal’s entry into these programs 
and his or her appointment as a principal, they saw the work of building bridges to early-stage program 
partners as less urgent than their work with principal-qualifying programs. 

Aligning District and Partner Priorities 

Our data collection in 2012-13 focused on what the districts designated as “preferred provider” programs, 
defined as those that not only incorporated the program features that The Wallace Foundation called for 



but also were working collaboratively with the district. Still, even among these collaborating programs we 
heard about some struggles for districts and their nonprofit or university partners in arriving at shared pri- 
orities. Not every partner was inclined — or able — to shift course content in response to new district priori- 
ties, such as the priorities that a new superintendent could bring. A simple example was that universities 
had to offer state-required courses and meet accreditation requirements, and thus could not comply fully 
with every request to tailor their program design to district specifications. A national nonprofit, New 
Leaders, had years of experience and research aimed at refining a program design and its criteria for partic- 
ipant selection and graduation, and the organization had to work with each partner district to negotiate 
these matters. Dynamics such as these will be discussed in a later chapter focusing specifically on partner- 
ships, but we note here that from a district’s perspective there was an incentive for the simpler option of 
starting or modifying its own program rather than working through a set of programmatic decisions with 
an independent partner. 


Summary 

The six districts were able to build on relatively strong preparation experiences already offered to aspiring 
principals, according to our survey results. With preparation in universities, alternate-route programs, and 
district offerings, the principals newly hired in the three years 2010-11, 2011-12, and 2012-13 reported 
that they were well prepared for their responsibilities in such key areas as instructional leadership. Their re- 
sponses also suggested that cohort models, problem-based learning, and clinical experience were wide- 
spread across preparation programs that served these districts before the grant award, although tailoring of 
content to the district was less so. Overall, their survey responses did not differ greatly from those of gradu- 
ates of exemplar programs surveyed by Darling-Hammond and colleagues (2007). 

The survey revealed other findings with important implications for the Principal Pipeline Initiative. It 
showed that almost five years was the median time elapsed from entry into preparation until placement as a 
principal; this placed pressure on the districts to strengthen the final stages of the preparation sequence in 
order to meet the grant requirement of filling vacancies with principals prepared in new ways by January 
2015. It also pointed to the large numbers of assistant principals who hoped to become principals, indicat- 
ing that work with assistant principals would also be key in this initiative. 

The districts’ approaches to bolstering their preparation pathways fell into three categories: 

Gwinnett and Hillsborough Counties continued to depend heavily on selective, district-run pro- 
grams, believing that these programs offered a good opportunity to select promising candidates 
and immerse them in the district’s leadership philosophy and practices. 

Denver and Prince George’s County moved closer to the model found in Gwinnett and 
Hillsborough Counties: they added selective, district-run programs as a final stage in preparation 
for assistant principals who had already gained the needed principal credentials through university 
programs or an alternate route. 
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Charlotte-Mecklenburg and New York City launched a wider portfolio of options among princi- 
pal-qualifying programs, adhering to a belief that individual candidates should have choices in pro- 
gram focus and design. New York, which had to prepare about four times more candidates than 
any other district (based on recent hiring data), expanded options of all kinds, including district- 
run programs as well as university and nonprofit programs at all stages of preparation. 

All districts moved to strengthen and formalize the learning opportunities offered to assistant principals. 
They recognized that most of these individuals wanted to become principals and that their learning was it- 
self part of the preparation pathway for principals. 

For all stages of preparation, district leaders saw their own preparation programs as having capacities not 
easily afforded by external programs. Their content and delivery were more easily customized to match dis- 
tricts’ leadership standards and other district priorities; district expertise could readily be accessed; and dis- 
trict hiring managers and other key staff could see firsthand the capabilities and growth of likely future 
candidates. 

With respect to the program features required in the RFP, every district was attending to each feature. 
Briefly summarizing the implementation status as of 2012-13, we can say: 

Principal-qualifying programs in each district engaged district and school staff in participant re- 
cruitment and selection, with all of the district programs and most of the external programs having 
direct participation by district staff. The extent to which selection was aligned with district stan- 
dards had reportedly increased at the time of our site visits. 

Cohort models were found in every district; they were present in all principal-qualifying programs, 
and all early-stage programs operated by districts or nonprofits. 

Program content was closely tailored to the district context in all district-run programs in 2013. 
The extent of tailoring was growing in external programs as well. 

Problem-based pedagogy was already reported by 72 percent of responding principals at baseline. 

It was very common in current program curricula as well, taking such forms as case studies of lead- 
ership, simulations, and projects to solve current school-based problems. 

Clinical experience was a feature of every program, but arrangements varied. Early-stage programs 
and half of the principal-qualifying programs gave participants leadership-related assignments to 
carry out in their current positions. The other half of principal-qualifying programs placed their 
participants in new positions and settings for periods ranging from a summer to a full school year. 
Whatever the arrangements — and all had advantages and disadvantages — district and program 
leaders were working with the principals in the buildings in which aspiring leaders were placed, 
trying to ensure an authentic leadership experience for the participants through some combination 
of selection, training, and monitoring of the mentor principals. 



Support for Novice Leaders 


I n contrast to the field of preservice preparation, support for new school leaders has been less studied 
and less regulated. Recent research on the ill effects of principal turnover on student performance 
(Seashore Louis, Leithwood, Wahlstrom, & Anderson, 2010) has brought renewed attention to the de- 
sirability of easing principals’ transition into the job so that promising leaders are not prematurely discour- 
aged. A component of the Principal Pipeline Initiative, therefore, is support for principals in their first three 
years on the job, linked to principal evaluation that identifies specific weaknesses to address. 

Although The Wallace Foundation expected support for novices to become a larger focus in the later years 
of the grant period rather than these earlier years, the six districts did not wait. We describe here the sup- 
port arrangements in place before and after the grant award, how novice principals and assistant principals 
perceived their support as of 2013, the districts’ approaches to adding and improving support, and the 
roles taken by partner organizations. 

In this area of work, like others related to principal preparation and support, terminology was inconsistent 
across districts. “Mentoring” and “coaching,” while used in distinct ways in some sites to identify a partic- 
ular type of help, did not have consistent definitions across sites. In this chapter we use “mentors or 
coaches” and “mentoring or coaching” when presenting cross-site findings but use the local term when an- 
alyzing the supports in a particular site. 


Before the Initiative 

Before the grant began, two districts, Charlotte-Mecklenburg and New York City, had substantial existing 
programs to build upon. Charlotte-Mecklenburg had a sequence of support designed to induct new princi- 
pals into their responsibilities, including group coaching for two years. In New York, the New York City 
Leadership Academy provided coaching for all first-year principals, and the district allowed principals to 
use school funds for further coaching if they chose to. 

The other four districts also offered mentoring or coaching support. Two disricts had formal systems: 
Gwinnett employed a cadre of recently retired principals on a part-time basis as “leader mentors” for 
novice principals; Prince George’s County engaged the National Association of Elementary School 
Principals to train experienced principals as mentors for new principals. Denver and Hillsborough County 
expected novice principals to receive support from their supervisors. 


Changes Made in Support for Novice Leaders 

The six districts readily agreed to bolster their arrangements for supporting new principals, as The Wallace 
Foundation required. All agreed to explore the use of tools or processes that had been developed with pre- 
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vious Wallace funding: the VAL-ED evaluation instrument, which gathers ratings and evidence of the prin- 
cipal’s performance on a series of measures based on school-leadership research (Goldring, Porter, Murphy, 
Elliott, & Cravens, 2009; Murphy, Elliott, Goldring, & Porter, 2007); and the School Administration 
Manager process for coaching principals in devoting more of their time to instructional leadership. 

As of 2012-13, each district had introduced changes in support for new principals and assistant principals. 
Each offered some combination of guidance from supervisors, nonsupervisory mentoring or coaching, pro- 
fessional development events or institutes, and professional learning communities or facilitated peer meet- 
ings designed to promote leader development. They had also undertaken changes in the monitoring and 
support roles of principal supervisors and in the extent to which evaluations of new principals were used to 
guide learning. 

Most notably, the districts instituted more intensive, individualized support to new principals, primarily 
through frequent, year-long mentoring or coaching but also through closer supervision. Five districts ex- 
panded or changed some aspect of their support for new principals after the grant award; New York City 
was considering some redesign of its system but had not made major changes as of spring 2013. The fol- 
lowing types of changes were made: 

Charlotte-Mecklenburg extended its program of formal support to five years. 

New coaching roles were added in Denver and Hillsborough County. 

The number of leader mentors grew in Gwinnett County. 

Denver, Hillsborough County, and Prince George’s County added more principal supervisors in 
order to allow closer working relationships between principals and their supervisors. 

New support was put in place to build the capacity of mentors, coaches, or supervisors in 
Charlotte-Mecklenburg, Denver, Hillsborough County, and Prince George’s County. 

Examples from Denver and Hillsborough County illustrate a multi-faceted approach to expanding and 
deepening mentoring or coaching support. 

Denver engaged a new, 20-member Executive Coaching Cadre to work with its first-year principals 
as well as with participants in its Learn to Lead preparation program. With backgrounds in a range 
of fields, including but not limited to education, the coaches were charged with helping develop the 
kinds of leadership skills that could be applied in any organization. According to interviewees at 
the district level, the support from these coaches was well received and, reportedly, coveted by vet- 
eran principals in the district. 

Hillsborough County developed a new coaching role and six full-time coaching positions to work 
with principals in their first two years. Coaches met weekly with new principals and biweekly with 



second-year principals. The district expected coaches and principal supervisors to coordinate and 
collaborate, such as by meeting jointly with the principal to set goals. Coaches also supported as- 
sistant principals in the district’s Preparing New Principals program, facilitating group learning 
among these individuals who had been identified as being on track for the principalship. The dis- 
trict also expects that over time the coaches will increasingly support principals in analyzing and 
planning the use of their time under the SAM process of the National SAM Innovation Project. 

Viewing assistant principals as potential future principals, two districts also added support for novices in 
the assistant principal position. As discussed in the previous chapter on principal preparation, Hillsborough 
County launched the Assistant Principal Induction Program in 2011, providing mentors and structured ac- 
tivities for assistant principals in their first two years on the job. Charlotte-Mecklenburg instituted a pro- 
gram in 2012-13 for new assistant principals, a program designed by the same group that had built the 
support sequence for new principals. 


Overview of Supports for Novice Principals in 2012-13 

The supports in place as of 2012-13 are described here, and they are contrasted with prior arrangements in 
Exhibit 11. 

Charlotte-Mecklenburg Schools 

The district developed a five-year sequence of support for new principals. Principals received coaching sup- 
ports in their first and second years, participated in external leadership training in their third and fourth 
years, and completed a capstone project in their fifth year. 

First- and second-year principals were matched with a Consultant Coach who met with and ad- 
vised principals in small groups. Beginning in 2011-12, second-year principals participated in the 
School Administration Manager (SAM) process. SAM coaches worked with principals on time use 
and instructional leadership. 

Third-year principals participated in the Executive Leadership Institute at Queens University of 
Charlotte. A collaboration between the university’s education and business schools, the program 
trained principals on leadership styles and organizational change. Fourth-year principals partici- 
pated in the Innovation Institute of the nonprofit McColl Center of Visual Arts, which focused on 
innovation for leaders from a variety of backgrounds. Finally, a fifth-year capstone project rounded 
out the support sequence. 

Denver Public Schools 

Novice principals received professional development and coaching. First-year principals had mentors, often re- 
cently retired principals. Beginning in 2012-13, principals were also assigned executive coaches to provide coun- 
sel on change management. Executive coaches were independent contractors with backgrounds in education and 
other fields. Denver also provided content-specific professional development monthly to all administrators. 


Policy Studies Associates, Inc. 



Cultivating Talent Through a Principal Pipeline 


Exhibit 11: Support for novice principals before and after grant award 


District 

Spring 2011 - Pre-grant 
(principals who receive support) 

Spring 2013 - Second year of grant 
(principals who receive support) 

Charlotte-Mecklenburg 

Consultant coach and cohort meetings 
(Y1&2) 

Consultant coach and cohort meetings (Y1&2) 

School Administration Manager (SAM) Innovation 
Project (Y2) 

Executive Leadership Institute at Queens 
University (Y3) 

Innovation Institute at McColl Center (Y4) 
Capstone project (Y5) 

VAL-ED evaluation for professional development 
(Y4&5) 

Denver 

Principal mentor trained by NYCLA (Y1) 

Principal mentor trained by NYCLA (Y1) 

Executive coach trained by NYCLA and University 
of Colorado (Y1) 

SAM Innovation Project (optional for all 
principals) 

Gwinnett County 

Assessment of novice skills and needs 
(Y1) 

Leader mentor (retired principal) and 
cohort meetings (Y1&2) 

Assessment of novice skills and needs (Y1) 

Leader mentor (retired principal) and cohort 
meetings (Y 1 &2) 

SAM Innovation Project (optional for all 
principals) 

Hillsborough County 

Urban Leader Institute with mentoring 
(principals in high-need schools) 

Urban Leader Institute with mentoring (principals 
in high-need schools) 

Principal coach trained by New Teacher Center 
(Y1 &2) 

Principal Induction Program (Y1&2) 

SAM Innovation Project (optional for all 
principals) 

New York City 

Principal coach trained by NYCLA (Y1) 

Option to continue principal coach 
using school funds (Y2) 

Principal coach trained by NYCLA (Y1) 

Option to continue principal coach using school 
funds (Y2) 

Prince George's County 

Principal mentor trained by NAESP 
(subset of novices) 

School Leaders Network (optional for 
all principals) 

Principal mentor trained by NAESP (Y1) 

Option to continue principal mentor (Y2) 

New Principals Academy monthly sessions (Y1&2) 

School Leaders Network (optional for all 
principals) 


Exhibit reads: As of spring 2011 (before The Wallace Foundation grant), Charlotte-Mecklenburg Schools provided consultant coaches and cohort meetings to principals in their first and 
second years as principals.. 

Note: Y1 denotes that the support was provided to first year principals, Y2 denotes that the support was provided to second year principals, etc. 

Source: Interviews conducted by the evaluation team and district reports to The Wallace Foundation 





Examples of Coaching and Mentoring 
Support for Novice Principals 

All six districts provided novice principals with for- 
mal coaching or mentoring support. Below are 
examples of strategies used: 

Individual goal setting with a support dyad of 
supervisor and coach/mentor 
Weekly 90-minute one-on-one support sessions 


Small cohort group professional learning com- 
munities 

"Executive coaching" focused on generalizable 
leadership behaviors 

Mentoring provided by trained, high-perform- 
ing. sitting principals 

Analysis of principal time use through the SAM 
Innovation project 

Non-evaluative feedback drawn from VAL-ED 
observations 


Gwinnett County Public Schools 

First-year principals interacted regularly with leader mentors, who were retired principals selected for this 
position by the superintendent. Second-year principals might also work with leader mentors based on need. 
Some principals participated in the SAM process and received coaching support on time management and 
instructional leadership. 

Hillsborough County Public Schools 


coaching for first-year principals, bi-weekly coaching for second-year principals, a summer institute, ten 
half-day sessions, and required courses. The number of principal supervisors, called Area Leadership 
Directors, was increased to permit them to spend more time with each principal. The district also modified 
support programs for assistant principals to improve alignment with expectations for principals. Finally, 
the district also collaborated with an external partner, the New Teacher Center, to improve support pro- 
vided by principal coaches and supervisors. 

New York City Department of Education 

The New York City Leadership Academy provided coaching for all first-year principals. Second-year princi- 
pals had the option of continuing this coaching through their school budgets. In addition, principals re- 
ceived support from their networks, into which schools self-select. 

Prince George's County Public Schools 

Novice principals and assistant principals received support including coaching and content-specific profes- 
sional development. All first-year principals were assigned a mentor principal trained by the National 
Association of Elementary School Principals. Second-year principals were offered the opportunity to con- 
tinue working with their mentors. Prince George’s County also worked with the School Leaders Network 
that provided monthly meetings focused on problems of practice for principals and assistant principals. 


New principals participated in the district’s two-year Principal Induction Program, which included weekly 
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How Partner Organizations Assisted with Support 


Every district engaged one or more partners in its efforts to strengthen support for novice leaders. Across 
the districts, a total of seven organizations served as partners in this work as of 2012-13 (Exhibit 12). Two 
of them partnered with multiple districts: the National SAM Innovation Project worked with four districts 
(with Charlotte-Mecklenburg, Gwinnett County, and Hillsborough County, and with Denver on a pilot 
basis); the New York City Leadership Academy worked on support in three districts (Denver, Gwinnett 
County, and New York). 


Exhibit 12: Number of partnerships for support, by district 


District 

Nearby universities 

Nonprofit organizations 

Total 

Charlotte-Mecklenburg 

2 

2 

4 

Denver 


2 

2 

Gwinnett County 


2 

2 

Hillsborough County 


2 

2 

New York City 


1 

1 

Prince George's County 


2 

2 


Exhibit reads: Charlotte-Mecklenburg partnered with two nearby universities on support. 


Mentoring or Coaching and Professional Development Provided Directly by Partners 

In some partnering arrangements, outside partner organizations provided support directly to new school 
leaders. The New York City Leadership Academy has had a contract to provide coaching to all first-year 
principals in New York City since prior to the Wallace grant. New Leaders provided support to its gradu- 
ates who served as novice principals. The National SAM Innovation Project directly trained and coached 
participating principals and their teams at start-up; over time, coaches employed by the district are ex- 
pected to take over a more prominent role, but support from the national project (including participation in 
its annual conference for selected teams) is expected to remain available. 

Two districts also asked partners to design and deliver professional development for novice principals. As 
part of its five-year sequence, Charlotte-Mecklenburg sent principals to Queens University’s Educational 
Leadership Institute. This series of training events, held over a six-month period, focused on helping princi- 
pals develop, implement, and re-evaluate change initiatives in their schools. Lacilitators helped the partici- 
pating principals work through protocols, based on the management literature, to assess school needs and 
plan change initiatives in their schools. Charlotte-Mecklenburg also sent principals to the Innovation 
Institute operated by a local nonprofit partner. Prince George’s County had a partner, the School Leaders 
Network, that helped organize and facilitate peer learning for new principals and assistant principals. The 
district also sent novice principals to some of the network’s professional development events. 






Capacity Building for Principal Coaches, Mentors, and Supervisors 

Districts engaged partner organizations to help build the district’s capacity to support new principals. Five 
partner organizations and institutions brought established track records in developing mentoring and 
coaching skills, and they worked behind the scenes to build these skills in the district staff working with 
principals. The New York City Leadership Academy has provided coaching on a large scale in New York 
for years, has developed protocols and training materials, and consults with school districts around the 
country on the development of mentoring programs. The New Teacher Center, a pioneer in programs of 
support for teacher induction, has also had programs in leadership coaching for years. The National 
Association of Elementary School Principals offers its own certification program for mentors of principals. 
The McColl School of Business at Queens University in Charlotte offers a master’s degree in executive 
coaching. The National SAM Innovation Project, with an extensive network of coaches who work with its 
participating school teams, has invested in systems for developing their coaching skills. 

Partner staff provided training institutes and onsite coaching to build the capacity of coaches, mentors, and 
supervisors. The partners’ work might focus on particular sets of leadership skills, notably instructional 
leadership and change management, that the district’s coaches and supervisors needed to help principals de- 
velop. Partners also brought expertise in coaching techniques and protocols. They work on teaching district 
supervisors and coaches to refrain from solving problems for novice principals and instead to ask the kinds 
of questions that would facilitate the principals’ on-the-job learning. 

New or expanded partnerships for novice-principal support in three districts grew out of the districts’ and 
partners’ previous experience working together: 

Hillsborough County contracted with the New Teacher Center, which has worked intensively with 
the district on coaching teachers, to help develop a principal coaching program and train the 
coaches. The New Teacher Center was involved in defining the role and responsibilities of the prin- 
cipal coaches and integrating the role with the district’s preservice and supervision components. 

The coaches had their own coaching and professional learning community, supported by the New 
Teacher Center. The aim was to help them learn “how to gather evidence of practice and give effec- 
tive feedback for practice,” according to a participant in designing these arrangements. At the same 
time, the New Teacher Center worked with principal supervisors, the Area Leadership Directors, in 
Hillsborough County to help build their skills in leader development. 

Charlotte-Mecklenburg received help with the coaching process from two universities with which it 
already had partner relationships. The director of the Leaders for Tomorrow preservice program at 
Winthrop University also had a major role in helping the district develop a coaching program and 
provided training for the district’s coaches. Queens University, in addition to offering its 
Educational Leadership Institute for principals, started a three-day academy for the district’s lead- 
ership coaches. Taught by an expert faculty member, the academy included role-playing exercises. 
The participants whom we interviewed described it as helping them bring new skills to their coach- 
ing role. 
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Under the grant, Prince George’s County expanded the participation of mentor principals in a for- 
mal program on mentoring techniques. The district sent mentor principals to the National 
Association of Elementary School Principals’ Leadership Immersion Institute program, where they 
developed their knowledge of adult learning and their techniques for “helping adults develop 
strengths to become effective leaders.” Mentor principals participated in training and a nine-month 
internship program in which they practiced providing feedback and support to new administrators. 

Alignment of Support with Evaluation 

All the districts reported in interviews that they wanted to align support with evaluation results and other 
principal-specific data, but that this work was still at an early stage as of 2013. Denver and Hillsborough 
County were working with a consulting firm, Cross & Joftus, on aligning professional development and su- 
pervisory support with evaluation results. As the districts build and populate data systems with data on in- 
dividual principals’ assessed skills and performance, they expect to do more in this regard. 


Support Reported by Novice Principals and Assistant Principals 

Our survey, administered in spring 2013, provides information on the supports that principals and assistant 
principals reported in 2012-13. Questions addressed whether they had a mentor or coach; the extent to 
which help from mentors, coaches, or supervisors helped them in various ways; professional development; 
and the feedback they received from principal evaluation systems. 

Across all the districts, survey respondents who were in their first year as principals in 2012-13 reported 
very high rates of formal mentoring or coaching support, while the extent of such support for second- and 


Exhibit 13: Percent of principals having mentors or coaches, 

by cohort, 2012-13 


District 

First-year principals 

(N=ioi) 

Second-year principals 

(N=i 38 ) 

Third-year principals 

(N=iop) 

Charlotte-Mecklenburg 

100 

100 

62 

Denver 

86 

30 

18 

Gwinnett County 

100 

85 

27 

Hillsborough County 

100 

100 

95 

New York City* 

100 

78 

63 

Prince George's County 

90 

63 

40 

OVERALL 

94 

73 

58 


Exhibit reads: Among first-year principals in Charlotte-Mecklenburg, 100 percent reported having a coach or mentor. 
*New York data should be viewed with caution due to the low response rate (31 percent). 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 






third-year principals varied by district (Exhibit 13). These responses were consistent with the district offer- 
ings described above: for example, Charlotte-Mecklenburg had formal coaching arrangements for all first- 
and second-year principals in 2012-13 and coaching as needed for third-year principals; Denver had coach- 
ing in place for first-year principals but not for most principals in their second or third years. At this time, 
Hillsborough County was the only district with principals reporting nearly universal principal coaching 
across all three years. 

Among first-year assistant principals, coaching and mentoring were common in two districts, Denver and 
Hillsborough County, but were reported by between 40 and 49 percent of respondents in the other districts 
(Exhibit 14). Only Hillsborough County — which added the Assistant Principal Induction Program as part of 
the grant — provided coaching to more than one-third of assistant principals in their second or third years. 


Exhibit 14: Percent of assistant principals having mentors or coaches, 

by cohort, 2012-13 



First-year APs 

Second-year APs 

Third-year APs 

District 

(N=i52) 

(N=i 38 ) 

(N=58) 


Charlotte-Mecklenburg 

49 

33 

17 

Denver 

82 

25 

33 

Gwinnett County 

48 

18 

11 

Hillsborough County 

100 

95 

86 

Prince George's County 

40 

25 

20 

OVERALL 

65 

35 

26 


Exhibit reads: Among first-year assistant principals in Charlotte-Mecklenburg, 49 percent reported having a coach or mentor. 
Source: Assistant Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 


Novice principals gave generally positive ratings to the support they received, whether from individuals for- 
mally designated as mentors or from others, such as their supervisors (Exhibit 15). Among all first-, second-, 
and third-year principals responding, 85 percent agreed or strongly agreed that they had support from indi- 
viduals knowledgeable about school leadership. About three-quarters (76 percent) agreed or strongly 
agreed that support had led them to make changes in their work. 

Assistant principals gave a less positive assessment of the support they received. The fact that assistant prin- 
cipals were less likely than principals to have mentors or coaches may account for some of the differences; 
the principals of the schools where they worked were probably the chief source of support for many assis- 
tant principal respondents. Notably, only about half of the assistant principals (51 percent) said they were 
supported in setting goals and developing an action plan for meeting them, indicating that districts could be 
doing more to help assistant principals chart their course for professional growth. 
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Exhibit 15: Principal and assistant principal perceptions of support 


The mentoring/coaching support 
1 have received this school year has: 

Percent of principals 
agreeing or strongly 
agreeing 

(N=352) 

Percent of assistant 
principals agreeing or 
strongly agreeing 

(N=3i8) 

a. Been provided by individuals who are knowledgeable about 
school leadership 

85% 

76% 

b. Been provided by individuals who are knowledgeable about the 
type of school 1 serve 

77 

71 

c. Addressed my specific needs 

76 

64 

d. Led me to make changes in my work 

76 

64 

e. Addressed the pressing issues in my school 

75 

59 

f. Helped me set effective goals and develop an action plan to reach 
those goals 

74 

51 

g. Been adapted or improved based on my feedback 

69 

53 

h. Helped me create or improve structures and strategies that 
support my teachers using student data to drive their instruction 

68 

57 

i. Used ongoing observation and data collection to monitor my 
(and my school's) progress towards my goals 

61 

45 

j. Helped me select professional development that meets my needs 

57 

47 


Exhibit reads: Eighty-five percent of first, second, and third-year principals agreed or strongly agreed that the mentoring/coaching they received from their area supervisor and/or 
mentor/coach had been provided by individuals who were knowledgeable about school leadership. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 


Districts might improve the ways in which mentors or coaches helped novice leaders choose professional 
development, according to the survey data. Compared with their assessments of other aspects of mentoring 
or coaching, both principals and assistant principals were relatively less likely to agree that mentoring or 
coaching had helped them select professional development that met their needs. 

When asked about their experience with professional development, however, principals’ and assistant prin- 
cipals’ responses were generally favorable. They tended to agree or strongly agree with positive statements 
about the professional development in which they participated, with 83 percent of principals and 84 per- 
cent of assistant principals agreeing that it had led them to make changes in their work (Exhibit 16). Areas 
of relative weakness in the professional development for new principals and new assistant principals, ac- 
cording to the survey results, were in helping them understand their students and their teachers. Unlike 
mentoring or coaching, where districts clearly made a greater investment in supporting principals than in 
supporting assistant principals, professional development was an area in which the assistant principals ap- 
peared to be receiving support that was comparable in perceived quality to what principals received. 





Exhibit 16: Principal and assistant principal perceptions 
of their professional development experiences 



Overall, my professional development experiences this year have: 

Percent of principals 
agreeing or strongly 
agreeing 

(N=343) 

Percent of assistant 
principals agreeing or 
strongly agreeing 

(N=3i9) 


a. Led me to make changes in my work 

83% 

84% 

b. Deepened my understanding of school leadership 

81 

83 

c. Included opportunities to work productively with principals from 
other schools 

74 

62 

d. Been sustained and coherently focused, rather than short-term 
and unrelated 

71 

76 

e. Included enough time to think carefully about, try, and evaluate 
new ideas 

68 

72 

f. Been closely connected to my school's improvement plan 

67 

74 

g. Helped me understand my teachers better 

64 

67 

h. Included opportunities to work productively with teachers at my 
school 

63 

73 

i. Helped me understand my students better 

58 

69 


Exhibit reads: Eighty-three percent of first, second, and third-year principals agreed or strongly agreed that the professional development experiences they participated in during the year 
led them to make changes in their work. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 


Connecting evaluation to support is a key tenet of the Principal Pipeline Initiative. Districts were making 
changes in their systems of principal evaluation, but in no site had these changes been completed in the 
2011-12 school year. Our spring 2013 survey asked principals about the communication of 2011-12 evalu- 
ation results because the 2012-13 evaluation process was not likely to have been completed at the time of 
the survey in the spring (Exhibit 17). 
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Exhibit 17: Feedback received by principals 
on the results of their performance evaluation 


Did you receive formal and/or informal feedback on the results of your performance 

Percent of respondents who 
were principals in 2011-12 

evaluation for LAST school year (i.e., 2011-12)? 

(N=213) 


a. Yes, written report 

78% 

b. Yes, required formal meeting with my evaluator 

55 

c. Yes, informal meeting with my evaluator 

19 

d. Yes, informal meeting with my mentor/coach 

16 

e. No, 1 received no feedback on my evaluation 

6 


Exhibit reads: Seventy-eight percent of second and third year principals reported that they received a written report summarizing the results of their evaluation from 2011-12. 
Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013. 


On the positive side, among respondents who had undergone evaluation in 2011-12, all but 6 percent had 
received the results. Formal written reports predominated as the means of communication, followed by for- 
mal meetings with an evaluator. A discussion of evaluation results with a mentor or coach was reported by 
only 16 percent of the respondents. 


Challenges Reported in Support for Novices 

Districts and their partners were addressing the areas of weakness that they perceived in their support for 
new principals and assistant principals. They recognized that the veteran principal supervisors and the vet- 
eran mentors or coaches charged with providing support needed help in two ways. First, not every long- 
time administrator was skilled in the kinds of instructional leadership that are expected from today’s 
principals. Second, these supervisors and mentors or coaches might also need help in effectively supporting 
adult learning. Districts brought in partner organizations to provide mentoring or coaching directly or to 
shore up the knowledge and skills of the district’s own supervisors and mentors or coaches. 

In three districts — Charlotte-Mecklenburg, Denver, and Prince George’s County — top leaders made the 
point in interviews that they wanted to support new principals in taking initiative and leading change, not 
just in complying with district directives as principals had expected to do in the past. They were trying to 
work with principal supervisors to ensure that this vision was clear in the support that principals received, 
but they acknowledged that it represented a change from the traditional culture of school districts and from 
the expectations that many principal supervisors were accustomed to meeting. 






Long-term funding for support could also prove to be a challenge. As of 2013, districts and partners were 
counting on the lasting benefits of their capacity-building efforts, hoping to have greater internal capacity 
to rely upon in the future. Some also mentioned hoping that systems of support and professional develop- 
ment would prove their worth and attract support, either in the district budget or from local philanthropy, 
and some were beginning to gather data to help in making the case for ongoing support in the future. 


Summary 

New kinds of supports were added for novice school leaders during the first two years of the grant period. 
As the districts continue to strengthen support for school leaders, they can build on assets reported here. 
These include the presence of mentors or coaches assigned to novice principals and assistant principals; the 
favorable overall assessments that principals and assistant principals offered when asked about their men- 
tors, coaches, or supervisors; and work the districts and partners have undertaken to build capacity in men- 
toring, coaching and supervision. 

Our survey results confirmed an observation that some district leaders made in interviews: the districts can 
improve the extent to which the components of support are connected to each other. For example, the link 
between mentoring or coaching and professional development might be strengthened. And as districts move 
forward in developing their evaluation systems, it will be critically important that they systematically con- 
nect evaluation feedback with support, as the Principal Pipeline Initiative requires. 
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Partnerships in Preparation and Support 


W orking relationships between districts and external partners are integral to The Wallace 

Foundation’s design of the Principal Pipeline Initiative. The Foundation urged participating dis- 
tricts to build a web of relationships with institutions and organizations that provide preservice 
and novice support programs, citing researchers’ view that such relationships make for more durable pro- 
grams in comparison with programs housed entirely within districts (Darling-FIammond et al., 2007; Orr 
et ah, 2010). The Foundation supported such partnerships with funding and technical assistance. 


In interviews with leaders in districts and partner programs, we gathered data on strengths and weaknesses 
of the existing partnerships, on approaches that they consider conducive to better working relationships, 
and on challenges in these relationships. 


District Stances Toward Providers 

A particularly influential study for The Wallace Foundation in crafting this initiative was that of Orr and 
colleagues, who classified districts’ arrangements for preservice preparation as reflecting the stance of a 
competitor, customer, or collaborator. This report has already discussed districts’ “competitor” stance as 
creators of their own programs for leader preparation. We will discuss here how districts wielded the power 
of customers while also collaborating with partner institutions and organizations. At this point in the study, 
we find the latter two stances to be intertwined in districts’ reported aims and actions. All six districts em- 
ployed more than one of the approaches identified by Orr and colleagues in the effort to drive changes in 
principal preparation. 

We found evidence everywhere of districts behaving as customers with partners and potential partners. For 
example, each of the districts set leadership standards that it will look for in program graduates. Each dis- 
trict also identified either a set of formal “preferred providers” or programs that it believed were most 
aligned with district interests. In all districts, the selection of programs to work with meant rejecting some 
programs. In some cases, we were told, districts declined to partner with local universities that were not 
philosophically aligned with the district’s beliefs about leadership competencies or how to develop them. In 
other cases, potential partners wanted limited communication: “We want to talk with you occasionally,” 
was how one district leader characterized the partnership offers that the district had rejected from some uni- 
versities. And some would-be partners expected to make few or no program changes, causing the districts to 
look elsewhere for partners. A district leader described the district’s newly strengthened stance, explaining 


...what we've done is taken that Wallace idea of district as consumer and said, we would love 
to work with you, "if." And those "ifs" are: you're willing to redesign the curriculum in align- 
ment with our [leadership] framework, you're willing to redesign your assessment systems to 
align with the framework, you'll participate in co-selection with us as a district. 
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At the same time, collaboration was also part of the work. Each of the districts has collaborated in some 
fashion — or at least discussed potential program improvement — with selected outside programs. We de- 
scribe here a rough typology of the working relationships we found. 

EQUAL PARTNERSHIP. True power-sharing partnerships were rare in these sites. The relationship be- 
tween Denver Public Schools and the Ritchie Program at the University of Denver was the longest-standing 
example. It was formally agreed to in 2003 by the chief executives of the district and the university. The 
university-based program leader attended weekly meetings in the school district, functioning as a member 
of the district team that worked on all components of the Principal Pipeline Initiative; classes were co- 
taught and the curriculum co-designed by that program leader and the district. 

COOPERATION AND NEGOTIATION. Communication and cooperation in partnerships were wide- 
spread across the districts. The district managers responsible for school leadership met with managers of 
partner programs; they informed them of district needs and expectations; and together they negotiated pro- 
gram features. A superintendent’s words captured the extent to which districts sought to set the terms of 
the programs, saying that productive partnerships have come from 


...really thinking long and hard on what our needs are, defining our needs. Whether on re- 
cruitment, selection, syllabus, content, faculty, process, internship, we defined our needs in all 
of those areas. With productive partnerships, they have worked with us to meet those needs. 

Similarly, another senior leader in the same district described a desirable partnership with a university part- 
ner as “shared ownership that is not [just] about repurposing existing content.” This interviewee expressed 
impatience with would-be partners who have said, in effect, “we’ve got this content [in our program], so 
we can just tweak it for you.” Instead, the district wanted to work with “someone who is committed to 
starting with a blank piece of paper and being your partner in co-creation, not tweaking what exists.” 

Something like the “blank piece of paper” has been considered a productive starting point in two other dis- 
tricts besides Denver, which used it more than a decade ago: 

Charlotte-Mecklenburg built preparation programs from the ground up with Winthrop University 
and then Queens University. 

New York City partnered in program development with the New York City Leadership Academy 
in the past decade; under the Principal Pipeline Initiative it again worked with the academy in 
building the new Leadership Advancement Program, and also worked with Relay Graduate School 
on design and development of a program that would launch in 2013-14. 

COMMUNICATION THAT HAS LED TO ADAPTATION IN PARTNER PROGRAMS. Working rela- 
tionships that included ongoing communication were common across the districts, and progress was visible 
in moving toward modification in partner programs, although it was not universal and not ordinarily rapid. 



In some contractual relationships, the district set terms on which it would work with an outside program, 
and the program offered some adaptation of the key components that it had developed. Working relation- 
ships with New Leaders took this form in Charlotte-Mecklenburg, New York City, and Prince George’s 
County. In each case New Leaders adapted its program to the local context in some ways, such as eligibility 
requirements and specific curricular content, while also holding fast to several fundamental program fea- 
tures. The New Teacher Center in Hillsborough and the National Association of Elementary School 
Principals in Prince George’s County adapted their national models of developing coaches’ and supervisors’ 
skills for novice support. For Prince George’s County, the National Institute for School Leadership tailored 
its existing leadership training programs in consultation with the district, adapting them to build the mod- 
ules for use in the Aspiring Leaders Program for Student Success. 

District leaders in Gwinnett and Hillsborough Counties described recent adaptations made in university 
programs to meet district expectations. Although Gwinnett and Hillsborough Counties did not fully depend 
on the universities for leader preparation, because they operated their own programs that supplied further 
preparation after candidates earned their university credentials, both provided input that resulted in at least 
some curricular or programmatic change in university programs, and both provided the services of district 
staff as instructors or coaches. 

Finally, little or no customization took place in Denver’s and Gwinnett County’s use of training modules for 
mentors from the New York City Feadership Academy, and the districts expressed satisfaction with the 
modules. In these cases the district was the customer purchasing services that followed an established 
model that the district was inclined to trust. 


Conditions Said to Facilitate Partnerships in the Sites 

In interviews, district staff and partner program staff alike identified a few practices and enabling condi- 
tions that they believe have contributed to beneficial partnerships. As senior leaders, they have seen some 
would-be partnerships of various kinds collapse in past years while others continue to operate, and that ex- 
perience has given them a basis for inferring what may have contributed to partnership under this initiative. 

Clarity in Roles and Responsibilities 

Interviewees said that having specific agreements brings useful clarity to roles and responsibilities. Some 
local preparation programs operated under long-term agreements that specified decision-making authority. 
The Ritchie Program in Denver was one example. Other examples were found in the university partner- 
ships with Charlotte-Mecklenburg: the institutions could not change features of their partner programs 
without permission from the district. New Leaders had contractual agreements with Charlotte- 
Mecklenburg, New York City, and Prince George’s County, subject to periodic renewal. Most partnerships 
had memoranda of agreement. 

District leaders and partners emphasized the importance of having clearly defined roles and responsibilities 
from the beginning of the partnership. District leaders described needing to communicate expectations re- 
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garding the division or sharing of responsibilities. For example, the New York City Department of 
Education learned through early experience in one partnership that it should “become much more clear on 
expectations and where the university owns the work, where we [both] share the work, and where we [in 
the department] own the work,” according to a district leader who had worked with the partner. Across the 
six sites, roles and responsibilities that had been divided or shared in partnerships included the recruitment 
and selection of candidates, curriculum development, teaching, selection of mentors for clinical experience, 
and training of those mentors. 

Frequent Communication Through Designated Channels 

As a district leader said, “occasional” communication was not enough for a partnership to be judged satis- 
factory. Communication had to be regularly scheduled, according to districts and partners. Interviewees in 
Charlotte-Mecklenburg, New York, and Prince George’s County mentioned having deliberately designated 
certain individuals as points of contact for their partners. Prince George’s County had a weekly check-in 
with its partner the National Institute of School Leadership. 

Problems have arisen when different district offices independently communicated priorities to a partner. We 
were told about instances in which this had happened, in two different districts. In each the two district of- 
fices reportedly had legitimate reasons to communicate with the partner, but the partner wasted valuable 
time in following instructions that were later overruled. 
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Accountability Around Clear Objectives 

A veteran district leader described the difference between meetings that produce tangible results and those 
that do not: 


[One lesson is] going into meetings with and coming out with deliverables. [In a meeting with 
a potential partner university] this morning, we had clear deliverables and owners. Because 
otherwise, my experience over thirty years is that you have these wonderful kumbaya meetings 
and then you come back a year later and you have another kumbaya meeting and nothing has 
really transpired. So out of our meeting this morning, right out of the box, we had somebody 
there taking notes about who owned the next step. I've already sent an e-mail to the people 
on our team saying when your deliverable is due. So really the lesson is having clear deliver- 
ables and establishing ownership, maintaining deadlines. It's a business relationship. It's not 
just what I would call a simple partnership that's loose. Keep it tight. 


To ensure that the agreed-upon plans would actually serve important purposes, district leaders said it was 
important for the district to be open about its needs and to ask for help. One said the district must “respect 
what [partners] have to offer,” and say, “’Our principals need ’X,’ so how, through your curriculum, can 
you help us?”’ 



Champions and Boundary Spanners 

Leaders of some partner programs previously worked in the district. This experience positioned them to 
serve two roles that are described as important in the partnership literature: they could be champions for 
the district’s vision within their institution; and they could be boundary spanners or “bridge leaders” who 
are able to communicate effectively within both the district and their own organization (Goldring & Sims, 
2005). A good example of such a leader was the director of Leaders for Tomorrow, who came to her posi- 
tion after 30 years of experience in Charlotte-Mecklenburg, including service as a principal and curriculum 
supervisor, and also carried out assignments for the district more recently. Sharing the district’s vision of 
leadership, this program director was able to craft a program that was closely tailored to district priorities 
and requirements. She was in regular communication with colleagues in the district, keeping abreast of new 
developments and adjusting the preparation program to meet new needs. 

Similarly, several partner programs working with New York City had leaders with high-level administrative 
experience in the New York City Department of Education; this was true of programs at the New York 
City Leadership Academy, Bank Street College, and Teachers College. The program leaders drew on their 
experience and collegial relationships within the district in trying to craft programs that fit district priorities 
and needs. 

Incentives for Partnership 

State certification requirements set some incentives for partnerships: when university programs are a 
mandatory part of the preparation pathway, a district has clear incentives to forge relationships with the 
universities that prepare many of its graduates and to try to influence their programs. As described in an 
earlier chapter, in each of the six states in which these districts are located, only universities and at most a 
few alternative providers were authorized to offer training that qualifies participants for licensure. This cre- 
ated an incentive for districts to engage with these providers. 

On the other side, when states require universities to offer clinical placements as part of their preparation 
programs, the universities must work with districts if only to arrange such placements. For example, 
Georgia required universities to develop memoranda of agreement with districts for clinical placements. 
States thus could give both districts and universities incentives to seek working relationships with each 
other, and similarly give bargaining power to the partner whose cooperation was needed. 

Disincentives are operating as well, and they impede some partner program features that districts may 
want. For example, making a university program more selective will diminish its tuition revenue in most 
cases. A department of educational leadership may feel that this is too high a price to pay for collaboration 
with a district. Accreditation systems and state requirements also constrained universities from drastically 
altering their program offerings. 

The Principal Pipeline Initiative, we should note, supplied incentives for partnerships: The Wallace 
Foundation expected the districts to forge partnerships with universities and other providers; the providers 
in turn could receive funding for their work in partnership with the districts. 10 Indeed, the grant brought 
not only funding incentives but also other enabling mechanisms to partnerships, and we turn next to those. 


10 The Wallace Foundation intends to launch a cost study of the Principal Pipeline Initiative, which will provide detailed analyses of the investments made and 
funding sources tapped; a cost analysis is not part of the present study. 
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Partnership Mechanisms in the Principal Pipeline Initiative 


Believing that strong partnerships will be important resources for the participating districts during and after 
the grant period, The Wallace Foundation incorporated several strategies into the initiative that are in- 
tended to foster such partnerships. These included funds that districts can choose to distribute to partner 
organizations and institutions, arrangements for the districts and selected partners to work together on 
projects initiated and supported by Wallace, and data systems that can enable a district to hold partners ac- 
countable for performance. 

Funding 

Grant funds in the Principal Pipeline Initiative can support start-up, expansion, or modification of partner 
programs through sub-awards from the recipient districts. Examples have been funding for start-up of the 
School Executive Leadership Academy at Queens University and for development of the program being de- 
signed at the Relay Graduate School of Education. At Bank Street College of Education and Teachers 
College, “Wallace Fellows” were specially selected participants in newly redesigned and expanded programs. 

The partner organizations providing help in connection with novice support were, for the most part, receiv- 
ing Principal Pipeline Initiative grant funds from the districts. An example was the National SAM 
Innovation Project, which had funding to work with Charlotte-Mecklenburg, Denver, Gwinnett County, 
and Hillsborough County. 

Planning for the sustainability of grant-funded offerings was part of the work of the pipeline districts, with 
guidance from Wallace. For example, with the National SAM Innovation Project, district staff were learn- 
ing to take on the coaching functions themselves and could take over all or part of the national project’s 
role in future years. Similarly, Prince George’s County’s robust relationship with the National Institute for 
School Leadership was designed to be of limited duration: once the modules for principal preparation have 
been developed and district staff have learned to oversee their use, the outside partner role will diminish or 
end. In some cases, too, partners were gathering evidence of effectiveness that they could use in seeking 
local philanthropic support to continue their work with the districts. 

The Quality Measures Process 

Another grant-supported vehicle for strengthening partnerships was collaborative use of the Quality 
Measures for Education Leadership Systems and Programs. Designed by Education Development Center 
with Wallace support, Quality Measures is a suite of tools that include indicators and rubrics for assessing 
leader preparation programs. It is organized around six program components that are associated with pro- 
gram effectiveness in leader preparation, drawing particularly on the work of Darling-Hammond and col- 
leagues (2007). The components are: course content and pedagogy; clinical practice; recruitment and 
selection; and three components related to graduate performance outcomes (knowledge, skills, and compe- 
tencies; responsiveness to market demand; and impact on school, teacher, student performance). 



The Principal Pipeline Initiative provided facilitators, training, and protocols for the Quality Measures 
work, which was intended to help districts and program leaders assess these program components in a 
data-based process. Leaders of programs in districts and partner organizations gathered evidence on the ex- 
tent to which their programs met criteria in each of the components listed; at each site, program representa- 
tives then met together with district staff and Quality Measures facilitators to review each program’s 
evidence and rate it on a rubric of indicators related to each component. 


As of early 2012, a total of 15 programs had been through Quality Measures. They included nine princi- 
pal-qualifying programs and four early-stage programs in principal preparation plus one program for 
novice support, and they represented a variety of home institutions. They included: 


District-run principal-qualifying programs from Gwinnett and Hillsborough Counties. 

Eight university-based preservice programs: half of which were principal-qualifying programs 
(three working with Charlotte-Mecklenburg and one with Denver); and the other half early-stage 
programs (two working with Gwinnett County and two with Hillsborough County). 


Three programs for preparation or support developed and led by other organizations: two that 
worked with Prince George’s County (the National Institute for School Leadership for principal- 
qualifying preparation, and the National Association of Elementary School Principals for support); 
and one from Denver (Get Smart Schools for principal-qualifying preparation). 

Two New Leaders principal-qualifying programs: one from New York City and one from Prince 
George’s County. 
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Across the 15 programs whose evidence was reviewed in early 2012, Education Development Center re- 
ported that the evidence of quality was best developed for course content and pedagogy, and least well de- 
veloped for all aspects of graduate program outcomes (Exhibit 18). Viewing these ratings as a baseline, the 
report went on to observe that program leaders had received action recommendations for their next steps 
in program improvement. An example, related to the program outcomes component, was: “Identify data 
points and develop mechanisms for tracking graduate performance and exchanging graduate data between 
school districts and program providers” (Education Development Center, 2012). 
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Exhibit 18: Aggregate team ratings for program components, 

2012, in percents 



Evidence-based ratings (N=ts) 

Program components 

Well- 

developed 

Developed 

Emerging 

Beginning 

Course content and pedagogy 

60% 

27% 

13% 

0% 

Clinical practice 

33 

53 

0 

13 

Recruitment and selection 

27 

27 

27 

20 

Graduate performance outcomes: 
Knowledge, skills, competencies 

40 

13 

7 

40 

Graduate performance outcomes: 
Responsiveness to market demand 

27 

40 

0 

33 

Graduate performance outcomes: 

Impact on school, teacher, student performance 

13 

13 

13 

60 


Exhibit reads: Of the 15 programs rated by local teams in the Quality Measures process, 60 percent were rated “well developed” with respect to course content and pedagogy on the 
basis of the evidence provided. 

Source: This table appears in Education Development Center, Inc. (2012, August). “QM baseline report of program quality ratings,” Watertown MA: Author, p. 6. 
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In assessing the effects of the Quality Measures process on both programs and partnerships, our interviewees 
pointed to the value of systematically reviewing and discussing program features and exchanging informa- 
tion. A typical positive comment was the following, from a leader of a university-based preservice program 
who compared the process favorably to traditional accreditation: 


We happen to think our program is pretty good, but we learned so much in the three-day 
process with other teams. We immediately started making refinements [to our program]. It was 
a really informative process. The protocols are so helpful in a way that is different from the ac- 
creditation reviews. 


Interviewees from two districts commented on the value of learning more about the operations of nearby 
university programs from the materials that the programs compiled. University participants from two sites 
commented that the process gave them a chance to learn about district policies and needs in the course of 
discussions around the table. 

Three interviewees commented on the amount of evidence to be gathered (what one called “our giant 
binders”), and one of these interviewees, based in a university, observed that there is no incentive for 
tenure-track faculty to devote time to this process: 





We have had concerns about the amount of time spent on Quality Measures. Several weeks of 
time. ... The type of work I am doing on Quality Measures and [with the district] will not be re- 
warded within a research university. How can I publish [when I'm doing this]? We still have our 
service commitments and research that are considered more important [in the university]. 


One district, Charlotte-Mecklenburg, deliberately planned to use Quality Measures as an opportunity to 
initiate a working relationship with a nearby university with which it had not yet worked closely. 
Interviewees in that district said that this intention was realized when the Quality Measures process 
“opened the door” to communication: 


[The university's] desire to develop a new program came directly out of the Quality Measures 
discussion. It was not because they assessed their current program and decided it was low-quality. 
[Rather] it opened the door for considering, how can we meet the needs of the district? 


Data and Data Use 

In addition to introducing the Quality Measures process as one avenue for the systematic compilation and 
communication of data, The Wallace Foundation is requiring each participating district to develop a leader 
tracking system. This system, which must conform to the Foundation’s specifications, will potentially en- 
able regular feedback loops regarding partner program operations and participants’ subsequent perform- 
ance. When fully developed and populated with data, the leader tracking systems will permit access to data 
spanning the entire career trajectories of leaders and potential leaders. One of many anticipated capabilities 
of the systems is the ability to tie an individual’s principal preparation experiences to data such as evalua- 
tion ratings and school performance. Over time, patterns in these relationships could inform program im- 
provement and contribute to program accountability. 
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Challenges in Partnerships 

Districts and partners discussed several challenges in forming and maintaining partnerships. At the most 
basic level, busy senior professionals had to make choices about investing time in partnerships and limit the 
number of relationships they would try to forge. A district staff member in Hillsborough County, while rec- 
ognizing that aspiring principals could and did earn their Level I certification in any of several universities, 
chose to focus on two institutions: 


To do a really deep partnership, which is what we're trying to accomplish, we didn't want to 
take on too many. Because we didn't think that we would have the capacity for the things 
we're trying to do with Nova and USF, which is meet regularly, exchange data, have them in- 
volved in our selection... So to have that kind of deeper partnership, we really did not want to 
bite off more than we could chew, and so we picked the two [universities] where the most can- 
didates [for assistant principalships] come from. 
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On the external partners’ side, similar trade-offs also arose around investing in a working relationship with 
a district, with the further complication that partners could be hesitant to tailor a program to one district 
when their graduates went on to serve in a number of districts. For example, one university representative 
expressed reservations about the idea of tailoring the program closely to a single district. This representa- 
tive was more enthusiastic about working with multiple districts to build an innovative program with more 
broadly applicable content. New Leaders, a partner with many districts nationally that invests in research 
and development around leader preparation, also had to manage the tension that could arise between 
maintaining its core program design and tailoring specific features to the context and priorities of each 
partner district. 

Turnover in leaders and staff, traditionally seen as a threat to durable partnerships, is a potential issue for 
partnerships in these sites as well. The leader of one partner program expressed some weariness with annu- 
ally re-introducing the program to new leadership in a district. Partner programs that worked primarily or 
exclusively with a single district might potentially find the change in priorities burdensome when turnover 
occurred. Since the start of the initiative in 2011, however, no partnerships have ended due to turnover, and 
no strong concerns about turnover were expressed in the interviews we conducted. 

A deeper challenge may lie in some built-in tensions between districts and university partners, stemming 
from state policies that tend to favor university preparation. These policies have, in the view of some dis- 
trict leaders, had the undesirable effect of forcing districts to rely on universities rather than their internal 
programs. As already discussed in this report, the five districts that operate their own preparation programs 
believe in those programs’ quality. In interviews, we heard serious skepticism from leaders in two districts 
about the need to work with universities in preparation. 

The length of the timeline for principal preparation, especially in some districts, also affected views on dis- 
trict-university collaboration. There were a few university programs whose graduates could become principals 
quickly: for example, the participants in the School Executive Leaders Academy whom Charlotte- 
Mecklenburg helped select in 2013 could potentially apply for principalships in 2014. In more cases, though, 
the payoff will be long in coming. The candidates selected in 2013 for a Level I certification program at a 
Florida university, for example, may become Hillsborough County principals someday, and therefore 
Hillsborough County is happy to help select them for this first-stage program and help to shape the course- 
work — but the earliest date when they could start on the job as Hillsborough County principals would be fall 
2020, under current arrangements. Similarly, Gwinnett County’s input in the admissions process of any of its 
partner universities in 2013 is relevant in shaping the potential principal cohorts of 2018 and beyond. For 
these two sites, collaboration with universities during the Principal Pipeline Initiative grant will have its effects 
on school leadership, if any, after the end of the grant period (and the evaluation period). 



Summary 


All six districts and all the programs for leader preparation or novice support discussed in this report put 
serious effort into forging partnerships. Although five districts had their own preparation programs, all the 
districts communicated and collaborated with external partners, and program adaptations resulted. When 
asked what facilitated their working relationships, interviewees in the sites pointed to some widely applica- 
ble principles of interorganizational interaction, such as clarity in roles and responsibilities, frequent com- 
munication through designated channels, accountability for meeting specific purposes, the presence of 
champions and boundary spanners who advocate for the partnership and help keep it on track, and under- 
lying incentives favoring partnership. 

The Principal Pipeline Initiative introduced other conditions that have been favorable to working relationships: 
not only did the initiative help pay for partners to work together, but it created specific vehicles for sharing in- 
formation about progress and results. With the Quality Measures process and, in the future, the availability of 
detailed data about leaders and their program participation, districts and their partners can look at program 
strengths and weaknesses together and, potentially, use these data to improve program results. 

Every partnership encountered at least some challenges, if only that of limited staff time. The interests of 
districts and partner institutions are not identical. In particular, districts’ desire for extensively customized 
programming could conflict with partners’ institutional interests in developing their own programs, meet- 
ing state or other accreditation requirements, and serving multiple districts. Still, much has been accom- 
plished through partnerships at this stage of the initiative. 
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Conclusions 


his report has addressed the implementation of two key parts of the Principal Pipeline Initiative, 
leader preparation and support for novices, during the initiative’s first two years. Based on research, 
The Wallace Foundation specified features of high-quality preparation and support and charged the 
six participating sites with putting those features in place. It communicated particular urgency about estab- 
lishing strong preparation programs quickly in order to improve the chances that the districts could hire 
substantial numbers of well-prepared principals by the fourth year of the initiative. We summarize here 
how the districts have fared in implementing the foundation’s charge. 


Reshaping the Preparation Sequence 

A theme in our findings is that implementation has reflected tension between the urgency of meeting grant 
requirements and the complexity of navigating preparation arrangements that involved multiple institu- 
tional partners and multiple stages of each individual participant’s career. 

The timeline for the initiative called on sites to rely on revamped preparation programs as a major source 
for the first-time principals to be appointed in 2014-15, three years after grant award. This expectation had 
implicitly assumed a relatively simple and rapid process of preparation. Sites would have one or two years 
to effect the needed improvements in preparation programs; aspiring principals would enroll in the im- 
proved programs in fall 2012 or 2013 and take office as principals soon after completing a program. In 
other words, a one- or two-year progression from program entry to installation as a principal was likely to 
be common. 

Our survey of novice principals shows, however, that the median time elapsed from starting preparation to 
assuming a principalship was almost five years. With a long principal preparation timeline, often involving 
multiple programs, all of the districts have actively worked to strengthen the programming that occurs at 
the end of that sequence. In five districts, this involved starting or modifying internal programs. Two dis- 
tricts launched their own new programs, the Learn to Lead residency in Denver and the Aspiring Leaders 
Program for Student Success in Prince George’s County, both designed for participants who could poten- 
tially be selected as principals in the following year. Since the grant began, three other districts fine-tuned 
existing district-run programs that are positioned at the end of the formal preparation sequence: Gwinnett 
County, Hillsborough County, and New York City worked to strengthen their internal programs whose 
graduates are allowed to apply for principalships. The one district that does not offer an internal program, 
Charlotte-Mecklenburg, has worked with two universities to launch new programs whose graduates could 
potentially be ready for principalships. 

Under the grant, three districts added to the number of years of structured preparation activity along the path- 
way to a principalship. Their intent was not that aspiring principals should attain that position more slowly, 
but that more formal preparation would take place in the years preceding the principalship. When Denver 
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and Prince George’s County added new programs for assistant principals who might become principals in the 
following year, these programs were over and above the state’s formal requirements for principal preparation. 
And at the earliest stage of preparation, Hillsborough County added a new Future Leaders Academy and 
Assistant Principal Induction Program, seeing both as prerequisites to an eventual principalship. 

The changes were not just about sequences, though. Districts and partners were building a portfolio of 
preparation offerings in each site, acting on strategies by which different programs could serve different 
purposes or populations. Charlotte-Mecklenburg and New York City, in particular, took a relatively flexi- 
ble stance on the preparation options for any individual candidate. Both of these districts worked with mul- 
tiple universities and one or more nonprofit providers to broaden the array of preparation options tailored 
to district standards and co-designed with district leaders. All districts exerted greater influence over prepa- 
ration through collaboration with external programs. They developed new or stronger partnerships around 
programs in which they shared the decision making. 

Districts were creating ways to follow the progress of principal aspirants and novice principals through stages 
of preparation and beginning practice, gathering data through their own and partner programs, and also 
working more intensively with aspirants. Internal gatekeeper programs gave district and school leaders first- 
hand knowledge of individuals who might soon be in the hiring pool. District and school leaders were also in- 
teracting with the participants in external programs, through participant selection, content delivery, and 
observation of clinical practice. As the districts worked with partners on preparation and novice support, they 
could track individuals’ progress, and new data-tracking systems will improve their ability to do so. This in- 
formation can potentially continue to inform redesign and fine-tuning of leader preparation in these sites. 


Features of Preparation 

In their approaches to strengthening leader preparation, the districts acted on different policy preferences. 
Gwinnett and Hillsborough Counties continued to stand out for their belief in a district-run pathway, be- 
ginning with individuals who hoped to become assistant principals and ending with hand-picked, high-po- 
tential candidates for a principalship in the next year or two; university preparation was also part of the 
pathway in each district, but in a lesser role due to a combination of state and district requirements. 
Charlotte-Mecklenburg and New York City worked to widen the options for aspiring leaders, cultivating a 
set of programs that differed in their focus and logistical demands so that a range of principal candidates 
might find the right pathway for them. Denver and Prince George’s County were moving in the direction of 
greater district control of the pathway, having initiated new programs for high-potential assistant princi- 
pals, but were also working with universities and nonprofits on principal-qualifying programs. In all cases, 
district leaders were collaborating with a range of external partners in preparation, as required. 

Whether by adding new programs or modifying existing ones (in large or small ways), the districts were 
aiming to implement the features of preparation programs identified as important in past research. As of 
2012 - 13 : 



District leaders conducted recruitment and selection for district programs and participated in these 
processes in some way for all the principal-qualifying programs operated by partners. 

Cohort models were universal among principal-qualifying programs. 

Problem-based learning had reportedly been part of preservice for most principals who were newly 
placed at or near the beginning of the initiative, according to our survey. Program leaders continue 
to work to integrate theory and practice. 

All districts were working on alignment of program content with their district standards and com- 
petencies for leadership. They had incorporated these into their own programming. They encour- 
aged partners to do the same, although programs that were subject to other requirements, such as 
the state requirements for certification programs, did not have complete flexibility to adhere to dis- 
trict standards. 


All district and partner programs featured clinical experience, although the arrangements varied. 
Half of principal-qualifying programs built learning opportunities into the participants’ current 
jobs, while the others offered placement in a new setting and position for periods up to a full 
school year in length. Many district leaders were of the opinion that the opportunities for authentic 
leadership responsibilities mattered more than duration or location, and all districts were working 
with the supervisors of clinical experience to build in such opportunities. 

Looking ahead, continuing to work on the quality of clinical experience during preparation was the fore- 
most challenge facing the districts and the programs with respect to program features. Cohort models and 
problem-based learning were firmly in place, as was a focus on instructional leadership and school im- 
provement; our survey findings corroborate the statements of program and district leaders on these points. 
Tailoring of content to the district was growing, although ultimately the extent of tailoring would have its 
limits in those university or nonprofit programs that were designed to serve more than one district. 
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Features of Support for Novices 

Support for novices, although a focus of less intense efforts at this stage of the Principal Pipeline Initiative, 
had strengths in the districts. The number and types of support programs had expanded. Mentors or 
coaches were already nearly universal for first-year principals, and available to many second- and third- 
year principals — and assistant principals — as well. Principals rated these individuals and their supervisors 
as generally knowledgeable and said that their own practice changed as a result of mentoring, coaching, or 
supervision. The assistant principals were less apt to rate their mentors, coaches, or supervisors highly, 
however. To bolster the capabilities of the mentors, coaches, and supervisors, every district has arranged for 
them to receive capacity-building support from partner organizations with relevant expertise. Districts 
pointed to fewer changes in the professional development that they offered to school leaders than in men- 
toring and coaching. Survey respondents gave generally positive ratings to their professional development. 


Policy Studies Associates, Inc. 



Cultivating Talent Through a Principal Pipeline 


Systems to link mentoring or coaching with professional development or with principals’ individual evalua- 
tion results were not fully developed across the sites as of 2013. According to the survey responses, it ap- 
pears that more could be done to help principals and assistant principals find relevant professional 
development and pursue their own plans for professional growth. This finding may change as the districts 
continue their work on developing and implementing evaluation systems for school leaders. 

Like the field of preparation, support for novices is complex and will take time to improve. It brings to- 
gether multiple systems within districts — mentoring or coaching, supervision, professional development, 
and evaluation. And outside partners, even though they have been helpful, add another layer of complexity 
to the challenges of rapidly building a coherent system. 


Partnerships 

Districts and partners alike said that partnerships had been mutually beneficial, particularly when the part- 
ners had clear roles and responsibilities, communicated regularly about agreed-on work, and had people 
who were internal champions for the working relationship. In addition, the Principal Pipeline Initiative 
brought several kinds of resources for partnership. These resources included not only funding but also 
greater access to data about program results, accompanied by technical assistance to help the districts and 
partners gather and use the data. 

Districts and their external partners faced a mixture of incentives and disincentives for working together. 
For example, state policy encouraged working relationships by authorizing selected universities and alter- 
nate routes to provide the preparation that would lead to principal certification, and by requiring universi- 
ties to offer clinical experience under formal agreements with school districts. At the same time, 
preparation programs were subject to accreditation requirements that made it cumbersome to modify their 
curriculum or selection requirements to fit district specifications. 

For partnership work as well as all the other work of the Principal Pipeline Initiative, time pressures 
weighed on the participants. This was true not only at a day-to-day level, with much work to be done, but 
also in considering what actions would bring rapid improvement to the principal corps and what actions 
might prove their worth over the longer term. For most districts, partnerships with universities around 
early-stage preparation programs fell into the latter category. Across districts, almost all interviewees in dis- 
trict leadership considered the work with universities to be a good investment in the preparation of future 
cohorts of principals. At the same time, five of the districts also valued their internal programs and were in- 
tent on strengthening them for both the short and the long term. 
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Appendix A: An Exploratory Comparative Analysis 


O ur principal survey affords an opportunity for an exploratory comparison of principals’ reports on 
their preparation with the findings of an important prior study (Darling-Hammond et ah, 2007), 
which helped inform the design of the Principal Pipeline Initiative. The prior study surveyed two 
groups of respondents: graduates of the programs highlighted as exemplars and a national comparison 
group of principals. Our survey replicates items from that study, allowing a comparison of the results. 

Caveats are in order for these comparisons. One relates to the time that has elapsed between the prior and 
current research. A national survey conducted in 2013 might well be expected to yield results different from 
those of respondents surveyed in the previous decade. It is likely that the preparation programs for princi- 
pals hired recently in the Principal Pipeline Initiative districts reflected current trends in school leadership 
practice, such as newly intensified expectations for data use. Moreover, they might reflect attention to the 
research on principal preparation that had not yet been reported a decade ago. However, the extent to 
which preparation programs approach or exceed the standard held up as exemplary in the prior research is 
of interest, and we will be able to report on it annually in this evaluation. 

With respect to assessments of program content and instructional approaches, and self-reported perceptions 
of their preparation for specific aspects of leadership, we found that the reports of novice principals in these 
districts were fairly similar to those of graduates of the exemplary programs studied by Darling-Hammond 
and colleagues, and somewhat more favorable than those of that study’s national sample. In comparisons 
where the effect size exceeds .50, we interpret the difference as at least moderate in size and therefore likely 
to be meaningful. Many of the differences that appear in these exhibits, although statistically significant, 
show small effect sizes; in these cases we do not interpret the apparent differences as being meaningful. 
Thus, for example, principals in our survey reported moderately greater emphasis on leadership for school 
improvement in their preservice than did Darling-Hammond’s national sample (d-. 71). On the other hand, 
we do not view the apparent difference in emphasis on leadership for school improvement between the 
principals we surveyed and the graduates of Darling-Hammond’s exemplary programs as meaningful; this 
is because, although the difference was statistically significant, the effect size was small (d=-.29). 

With those caveats and suggestions for interpretation, we show in Exhibits A-l through A-3 the compar- 
isons between our survey results and those of the prior study. 
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Exhibit A-1: Principal perceptions of preservice content 


To what extent were the 
following qualities/practices 
true of your formal leadership 
preparation training? 

PPI Evaluation 
Surveys in 2013 

Darling-Hammond et al. data reported in 2007 

Principal 

(n= 3 i 8 ) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(H= 6 ig) 

Mean 

Effect Size 

Mean 

Effect Size 

a. Program content emphasized 
leadership for school 
improvement 

4.24 

4.49** 

-.29 

3.63** 

.71 

b. Program content emphasized 
instructional leadership 

4.23 

4.58** 

-.41 

4.13* 

.12 

c. Program gave me a strong 
orientation to the principalship 
as a career 

4.01 

4.39** 

-.43 

3.73** 

.32 


Exhibit reads: Principal responses averaged 4.24 on the extent to which program content emphasized leadership for school improvement. Darling-Hammond et al. exemplary 
programs responses averaged 4.49 and the national comparison responses averaged 4.39. Response means in this table are on a five-point scale in which 1 equals “not at all” and 
5 equals “to a great extent.” 

Note: T-tests were conducted to determine statistical significance; *p<.05 and **p<.01. Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/- 
.50 as a minimum threshold for interpreting results as having at least a moderate effect. 


Leadership Institute. 
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Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013; Principal surveys reported in Darling-Hammond, L., LaPointe, M., Meyerson, D., Orr. M.T., & 
Cohen, C. (2007). Preparing school leaders for a changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational 




Exhibit A-2: Principal perceptions of preservice coursework 


To what extent were the 
following leadership 
practices/instructional 
strategies part of your 
coursework in your leadership 


PPI Evaluation 
Surveys in 2013 

Darling-Hammond et al. data reported in 2007 


Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 p) 


Principal 


preparation training? 

(n=3i8) 

Mean 

Effect Size 

Mean 

Effect Size 

a. Analysis and discussion of field- 
based problems/problem-based 
learning approaches 

3.88 

4.29** 

-.34 

3.47** 

.34 

b. Analysis and discussion of case 
studies 

3.88 

4.39** 

-.43 

3.74* 

.12 

c. A portfolio demonstrating your 
learning and accomplishments 

3.78 

4.36** 

-.44 

2.81** 

.73 

d. Field-based projects in which 
you applied ideas in the field 

3.77 

4.22** 

-.36 

3.37** 

.32 

e. Action research or inquiry 
projects 

3.62 

4.00** 

-.31 

3.34** 

.23 


Exhibit reads: Principal responses averaged 3.88 on to what extent analysis and discussion of field-based problems/problem-based learning approaches was a part of their 
coursework. Darling-Hammond et al. exemplary programs responses averaged a 4.29 and the national comparison responses averaged a 3.47. Response means in this table are on 
a five-point scale in which 1 equals “not at all” and 5 equals “to a great extent.” 

Note: T-tests were conducted to determine statistical significance; *p<.05 and **p<.01. Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/- 
.50 as a minimum threshold for interpreting results as having at least a moderate effect. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013; Principal surveys reported in Darling-Hammond, L., LaPointe, M., Meyerson, D., Orr. M.T., & 

Cohen, C. (2007). Preparing school leaders for a changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational 
Leadership Institute. 
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Exhibit A-3: Principal perceptions of their preparedness 


How effectively did your 
leadership preparation 
training prepare you to do 
each of the following: 

PPI Evaluation 
Surveys in 2013 

Darling-Hammond et al. data reported in 2007 

Principal 

(n=3i8) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 p) 

Mean 

Effect Size 

Mean 

Effect Size 

a. Engage in self-improvement and 
continuous learning 

3.36 

3.58** 

-.30 

2 9i ** 

.60 

b. Create a collaborative learning 
organization 

3.14 

3.34** 

-.25 

2.69** 

.57 

c. Use data to monitor school 
progress, identify problems and 
propose solutions 

3.13 

3.31** 

-.21 

2.47** 

.75 

d. Evaluate teachers and provide 
instructional feedback to 
support their improvement 

3.08 

3.36** 

-.33 

2.82** 

.30 

e. Engage staff in a decision 
making process about school 
curriculum and policies 

3.07 

3.22** 

-.18 

2.70** 

.44 

f. Lead a well-informed, planned 
change process for a school 

3.05 

3.22** 

-.21 

2.59** 

.56 

g. Engage in comprehensive 
planning for school 
improvement 

3.04 

3.22** 

-.21 

2.58** 

.56 

h. Create a coherent educational 
program across the school 

2.90 

3.22** 

-.39 

2.63** 

.33 

i. Redesign school organizations 
to enhance productive teaching 
and learning 

2.88 

3.06** 

-.20 

2.46** 

.49 

j. Design professional 
development that builds 
teachers' knowledge and skills 

2.84 

3.25** 

-.47 

2.50** 

.39 

k. Understand how different 
students learn and how to teach 
them successfully 

2.68 

3.03** 

-.40 

2.56* 

.14 

1. Evaluate curriculum materials 
for their usefulness in 
supporting learning 

2.50 

2.92** 

-.45 

2.49 

.01 


Exhibit reads: Principal responses averaged 3.36 on how prepared they were to engage in self-improvement and continuous learning. Darling-Hammond etal. exemplary programs responses averaged 3.58 
and the national comparison responses averaged 2.91. Responses on this table are on a four-point scale in which 1 equals “not at all” and 4 equals “very well.” 

Note: Response means from the Darling-Hammond et al. survey were converted from a five-point scale to a four-point scale to allow for comparisons. 

Note: T-tests were conducted to determine statistical significance; *p<.05 and **p<.01. Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/- 
.50 as a minimum threshold for interpreting results as having at least a moderate effect. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013; Principal surveys reported in Darling-Hammond, L., LaPointe, M., Meyerson, D., Orr. M.T., & 

Cohen, C. (2007). Preparing school leaders for a changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational 
Leadership Institute. 
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